https://github.com/elastic/elasticsearch

sort by:
Revision Author Date Message Commit Date
2c11a3f Merge remote-tracking branch 'upstream/main' into feature/semantic-text 04 April 2024, 08:45:57 UTC
7b25421 Add ES|QL signum function (#106866) * Add ES|QL signum function * Update docs/changelog/106866.yaml * Skip csv tests for versions older than 8.14 * Reference layout docs file and fix instructions for adding functions * Break csv specs by param type * More tests 04 April 2024, 07:48:35 UTC
996a164 Set visibility of failure_store param of Rollover API to feature_flag (#107061) 04 April 2024, 07:47:14 UTC
daa9006 Apply snapshot `?after` filter inline (#107003) In `TransportGetSnapshotsAction` today we build a list of all candidate snapshots and then copy them into another list to apply the `?after` filter. With this commit we construct the final filtered list directly. 04 April 2024, 07:17:14 UTC
9a2f8a8 Add remote cluster network troubleshooting docs (#107072) Spells out in a little more detail our expectations for remote cluster connections, including an example log message when the network is unreliable and some suggestions for how to troubleshoot further. 04 April 2024, 06:37:13 UTC
72a8248 [Profiling] Use default task cancellation check (#107037) With this commit we remove our custom implementation of whether a task has been cancelled and instead use the standard implementation that is already provided by the task API. 04 April 2024, 05:05:18 UTC
a37debd Ensure getUser() is the logical user, not API key creator for RCS 2.0 (#107023) This commit changes SecurityContext#getUser() to provide the original user that initiated the call when run across clusters for RCS 2.0. Before this change the getUser() would provide the RCS 2.0 API key creator as the current user. 03 April 2024, 21:40:41 UTC
147f5a0 ESQL: Introduce language versioning to REST API (#106824) For the _query endpoint, add a parameter for the ESQL language version to the JSON payload. For now, it is optional and is only validated with no further action. 03 April 2024, 19:23:51 UTC
f49ead7 [ML] Add Cohere rerank to _inference service (#106378) 03 April 2024, 18:48:22 UTC
89563c2 Fix auto sharding recommending 0 shards for 0.0 workload (#107064) 03 April 2024, 17:27:13 UTC
8cab439 Mark enrich stats API as internal instead of public (#107022) This was previously publically available, however it exposes node counts and IDs. After discussion we've decided it's unnecessary on Serverless. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> 03 April 2024, 17:15:48 UTC
42df8fe [Inference API] Make completion task available in inference processor (#107060) 03 April 2024, 17:02:24 UTC
d88836b [ML] Add missing TokenizationConfigUpdate named writable to registry (#107056) 03 April 2024, 16:19:23 UTC
2f33b01 Split the mappings for failure stores out of the index template service. (#107025) This will help us keep failure store specific configurations in one place, and hopefully make it easier to evolve the schema in the future. 03 April 2024, 16:08:48 UTC
245d69d [Security Solution] Add `read` permission for third party agent indices for `kibana_system` (#107046) * add read permission for third party agent indices Allow `read` privilege for `kibana_system` role on `logs-sentinel_one*` and `logs-crowdstrike*` index patterns closes elastic/security-team/issues/9046 * Update docs/changelog/107046.yaml * review changes 03 April 2024, 15:53:38 UTC
36bcb6b Query API Keys support for both `aggs` and `aggregations` keywords (#107054) The Query API Key Information endpoint supports aggs since #104895. But some lang clients actually use the `aggregations` keyword in requests, as the preferred synonym to `aggs`. This PR adds support for the `aggregations` request keyword as a synonym for the existing `aggs` term. Closes #106839 03 April 2024, 15:33:14 UTC
755226d [TEST] Add full cluster restart test for downsampling (#107053) This is a copy of the test added for rolling upgrade in #107036 03 April 2024, 15:09:17 UTC
6e36089 Add non-indexed fields to ecs templates (#106714) * add non-indexed fields to ecs templates * update * Update docs/changelog/106714.yaml * Adjusting ECS tests to verify proper index and doc_values settings * review comment --------- Co-authored-by: eyalkoren <41850454+eyalkoren@users.noreply.github.com> 03 April 2024, 15:04:24 UTC
f9ae6db ESQL: Add docs for the OPTIONS directive (#107013) This adds the docs for the newly added `OPTIONS` directive to `FROM`. 03 April 2024, 14:23:36 UTC
377c7e5 Implement time series grouping via HashAggregationOperator using a specialized block hash implementation (#106127) This change adds new operator factory that performs grouping by `_tsid` and `@timestamp` field. The new operator factory delegates to the `HashAggregationOperator` do the grouping, but does make use of a specialized block hash (which makes a few assumptions about the group fields). This is an initial implementation that is not optimize for 'local' grouping. The new time series grouping operator factory isn't usable from API and is only usable from tests. 03 April 2024, 14:01:50 UTC
333c6a6 Replace UnsupportedOperationException with IllegalArgumentException (#107038) This makes sure the Rest status we return is a 4xx instead of a 5xx, which means the error does not map anymore to "Service Unavailable". 03 April 2024, 13:58:57 UTC
206a0b7 [DOCS] Remove obsolete accounting circuit breakers (#107015) 03 April 2024, 13:54:53 UTC
ea9e6a9 Clarify closed index setting warning (#106888) 03 April 2024, 13:48:31 UTC
b97e2d6 ESQL: Fixup docs for LOG and LOG10 (#106963) This merges all of the hand written docs for `LOG` and `LOG10` into the annotations which updates the `META FUNCTIONS` - now it'll always be the same as the docs. This also deletes the hand maintained docs and let's the documentation generation process rebuild it. 03 April 2024, 13:46:32 UTC
8dff321 Remove allowedFields argument from SearchExecutionContext constructor (#107021) allowedFields is a mutable instance member, that only ever gets set via its setter. It does not need to be provided via constructor. Removing a needless argument seems important given the number of other arguments already present. 03 April 2024, 13:46:03 UTC
a3a60b5 [TEST] Add rolling upgrade test for downsampling (#107036) Simple test, sets up downsampling to run in the old cluster, then waits for it to complete and verifies that downsampled indexes can get queried in the mixed and upgraded clusters. 03 April 2024, 13:21:41 UTC
febc23c [Transform] Clarify afterFinishOrFailure javadoc (#107048) Close #100515 03 April 2024, 13:15:19 UTC
bb520eb [Transform] Implement robustness test that constantly creates/starts/stops/deletes a continuous transform (#106313) 03 April 2024, 13:07:48 UTC
1c923a6 [ML] Convert DatafeedContext to a Record (#107009) Refactor DatafeedContext from a POJO with a Builder to a Record. This is mostly to reduce code footprint and improve readability in the `DatafeedContextProvider` class. The lambda closures are now responsible for passing the fields to the DatafeedContext, rather than the Builder. Close #107008 03 April 2024, 11:46:12 UTC
c74490c ESQL: Enable VALUES agg for datetime (#107016) When I wrote the `VALUES` agg I didn't plug it in for `datetime` fields. Ooops. We just have to plug it in. 03 April 2024, 11:42:40 UTC
aa03bc8 ESQL: Swap `FROM` `METADATA` and `OPTIONS` directives (#107012) This swaps the two clauses, foreseeing a more frequent use of METADATA one, over OPTIONS. 03 April 2024, 11:30:09 UTC
a23e29c [Test] Explicitly specify number of shards to 1 (#107032) The test assumes number of shards defaults to 1 which may not true in certain deployment types. This PR makes it explicity configure it to 1. Relates: #106707 03 April 2024, 09:34:58 UTC
331d783 [DOCS] Fixes search request in semantic search tutorial. (#107034) 03 April 2024, 09:30:14 UTC
0c493e4 Make requestSize final in SearchExecutionContext (#107026) 03 April 2024, 09:21:06 UTC
573c032 [Docs] Fix CCS matrix for 8.13 (#107028) 03 April 2024, 08:54:49 UTC
99138c1 [Failure store] Refactoring data stream lifecycle to facilitate managing failure store. (#106981) In this PR we refactor data stream and data stream lifecycle related code. The goal of this small refactoring is to make it easier to extend DSL to manage failure store. - The retrieval of backing indices past the retention period. - The rollover execution during the DSL run to handle more than one write index. 03 April 2024, 08:14:47 UTC
80bba1a ESQL: Use SearchExecutionContext in SearchStats (#106999) This is a lighter dependency than requiring SearchContext. 03 April 2024, 07:58:52 UTC
431f73f [TEST] Wait for downsampling op to complete on master side (#106918) * unmute test, remove failing assert * Wait for downsampling op to complete on master side * read asserted value properly * add invalid config status 03 April 2024, 06:02:23 UTC
f930544 Introduce ordinal bytesref block (#106852) This PR introduces ordinal-based blocks for BytesRef. These blocks consist of a pair: an IntBlock for ordinals and a BytesRefVector for the dictionary. Compared to the regular BytesRefBlock, these blocks are slower due to indirect access and consume more memory because of the additional ordinals block. However, they offer significant speed improvements and reduced memory usage when byte values are frequently repeated. There are several use cases where these blocks can be beneficial. 03 April 2024, 02:35:48 UTC
e6ccaf4 ESQL: Split the tests for META FUNCTIONS (#106954) This splits the tests for META FUNCTIONS into a few tests that fit better on the screen. There's an interesting bug in the CSV parser where if you end the last line with an empty string it'll consider it `null` which we think of as `null`. I bumped into that and it's caused me some trouble. I had to work around it here. Thus the assertion changes. Basically `[]` will force us to parse a list. So we can use `[""]` - a list of just the empty string - to force the testing framework to spit out an empty string. We only need this because the empty string is the last item in the list.... 03 April 2024, 00:42:14 UTC
e39fd58 Fix jvm ergonomics tests (#106969) closes #106554 02 April 2024, 20:40:33 UTC
1e253a0 Propagate ?master_timeout query parameter from CCR apis to downstreams (#105168) * Propagate ?master_timeout query parameter from crr put follow api * Update docs/changelog/105168.yaml * fix sily mistake * spotless * fix 2 * Add ?master_timeout query param to apis which use it * Update rest api specs + yaml tests * Add master_timeout (+timeout) to remaining api endpoints * Update 105168.yaml Correct message * Enable randomly ?master_timeout param to ccr IT tests * Add timeout param to RestCcrStatsAction * propagate master_timeout param between put_follow -> resume_follow calls * Propagate master_timeout down to persistent task layer * Add transport version for ccr stats request object change * Add BwC test for CcrStatsAction.Request obj 02 April 2024, 20:39:01 UTC
412f2f5 New Lucene spanish plural stemmer (#106952) Add Lucene SpanishPluralStemFilter based on apache/lucene#11284 02 April 2024, 19:53:19 UTC
646ac4f Update license acknowledgement messages for Search (#106857) License state messages are shown to user when license expires, or license state changed. Updated Search text to include current features. 02 April 2024, 17:20:39 UTC
ee7e254 Remove remaining Version usages in licensing (#101800) This commit is a follow-up to the changes converting the trial license refresh logic to use `TrialLicenseVersion` (#100169), converting the remaining uses of `Version`, which are used to ensure wire compatibility, with `TransportVersion`. Incidentally, this commit also removes code for compatibility with 7.6.0 and earlier, as those versions can no longer be in a cluster with current-version nodes. 02 April 2024, 17:14:41 UTC
08c7b65 Remove `ClusterStateObserver` constructor with implicit timeout (#107004) This constructor is only used in tests, and it's trappy to hide the timeout from callers like this. Inlining it to make the timeout explicit everywhere. 02 April 2024, 15:14:22 UTC
b6ca8d2 [feature/semantic-text] semantic text copy to support (#106689) 02 April 2024, 14:38:18 UTC
499ebbf ESQL: Fix issues with ST_* functions and compound geometries (#106992) * Add mores tests to cartesian complex geometry tests * Support multi-shapes within constant literals This was not supported for lucene push-down for any of the relates functions on point indices: ST_INTERSECTS, ST_WITHIN, ST_CONTAINS. In addition ST_CONTAINS had specific bugs around handling multi-component shapes within literals/constants. * Update docs/changelog/106992.yaml * Delete docs/changelog/106992.yaml * Control result order for multi-cluster tests 02 April 2024, 14:36:56 UTC
e8a3c9b AwaitsFix #106968 02 April 2024, 14:12:42 UTC
225bb2c Inject reserved role name checker into role mgmt actions (#106759) This PR makes reserved role name validation injectable for the create and delete roles actions. 02 April 2024, 13:58:13 UTC
959ef23 Accumulate results directly in TransportGetSnapshotsAction (#106983) There's no need to return a `SnapshotsInRepo` object all the way along the chain of listeners, we can just collect the results directly and signal completion with a `Void` response. 02 April 2024, 13:51:56 UTC
d8a9046 ESQL: Fix VALUES test for `text` (#106955) This fixes an error in the test for `text` fields running the `VALUES` agg. We have to sort the output or we'll get it in a random order. Closes #106939 Closes #106894 02 April 2024, 13:45:35 UTC
3c10d47 Prune changelogs after 8.13.1 release 02 April 2024, 13:38:50 UTC
35b7212 Bump versions after 8.13.1 release 02 April 2024, 13:37:41 UTC
c9ab203 Remove AwaitsFix 02 April 2024, 13:22:54 UTC
f2e594f Remove unused XContent parsing logic (#106950) Another round of mostly automatically cleaning up unused xcontent parser methods and parser instances that become unused as a result. 02 April 2024, 13:18:54 UTC
ee6ff33 ESQL: Remove unused test class (#106996) We don't use it any more. 02 April 2024, 12:46:41 UTC
480ee1c Convert FetchFieldsContext to a record (#106991) Simple refactoring to save a few lines of code. 02 April 2024, 12:38:59 UTC
e56dcee [Inference API] Add completion task type docs (#106876) 02 April 2024, 11:34:46 UTC
7f17eff Implement rollover for failure stores (#106715) Allows rolling over failure stores through the existing RolloverRequest by specifying `IndicesOptions.FailureStoreOptions` on that request. 02 April 2024, 11:03:35 UTC
2380492 ESQL: Support ST_CONTAINS and ST_WITHIN (#106503) * WIP Started adding ST_CONTAINS * Add generated evaluators * Reduced warnings and use correct evaluators * Refactored tests to remove duplicate code, and fixed Contains/multi-components * Gradle build disallows using getDeclaredField * Fixed cases where rectangles cross the dateline * Fixed meta function tests * Added ST_WITHIN to support inverting ST_CONTAINS If the ST_CONTAINS is called with the constant on the left, we either have to create a lot more Evaluators to cover that case, or we have to invert it to ST_WITHIN. This inversion was a much easier option. * Simplify inversion logic * Add comment on choice of surrogate approach * Add unit tests and missing fold() function * Simple code cleanup * Add integration tests for literals * Add more integration tests based on actual data * Generated documentation files * Add documentation * Fixed failing function count test * Add tests that push-to-source works for ST_CONTAINS and ST_WITHIN * Test more combinations of WITH/CONTAINS and literal on right and left This also verifies that the re-writing of CONTAINS to WITHIN or vice versa occurs when the literal is on the left. * test that physical planning also handles doc-values from STATS * Added more tests for WITHIN/CONTAINS together with CENTROID This should test the doc-values for points. * Add cartesian_point tests * Add cartesian_shape tests * Disable Lucene-push-down for CARTESIAN data This is a limitation in Lucene, which we could address as a performance optimization in a future PR, but since it probably requires Lucene changes, it cannot be done in this work. * Fix doc links * Added test data and tests for cartesian multi-polygons Testing INTERSECTS, CONTAINS and WITHIN with multi-polydon fields * Use required features for spatial points, shapes and centroid * 8.13.0 is not yet historical version This needs to be reverted as soon as 8.13.0 is released * Added st_intersects and st_contains_within 'features' * Code review updates * Re-enable lucene push-down * Added more required_features * Fix point contains non-point * Fix point contains point * Re-enable lucene push-down in tests too Forgot to change the physical planner unit tests after re-enabling lucene push-down * Generate automatic docs * Use generated examples docs * Generated examples use '-result' prefix (singular) * Mark spatial functions as preview/experimental 02 April 2024, 08:31:00 UTC
20fc2d2 Test modifying and removing data in ProfileIT (#106582) User Profiles can be used to store application data against a user (e.g. user preferences). This commit extends the integration tests for profile date storage to include explict tests for partial updates and clearing existing data 02 April 2024, 08:27:33 UTC
7ee63df Expand docs on reset desired balance API (#106921) Explains its purpose and surrounding context a little more, including a note that this should never be necessary (i.e. if you find you need it, that's a bug). 02 April 2024, 08:18:59 UTC
519580e AwaitsFix for #106957 02 April 2024, 07:42:19 UTC
0eca03c Remove unused `SnapshotsInRepo#remaining` (#106674) We only discard snapshots using the `?size` parameter when constructing the final response, so we can count the `remaining` snapshots in a local variable rather than tracking an unnecessary `0` on every `SnapshotsInRepo`. Indeed by inlining `sortSnapshots` into the routine that constructs the final response we can avoid even having to build the a final `SnapshotsInRepo` here. 02 April 2024, 06:52:25 UTC
89cfb85 [Test] Fix AsyncSearchSecurityIT testStatusWithUsersWhileSearchIsRunning (#106912) The error_query is only available in snapshot builds. All test failures have the release-tests tag. Closes #106871 02 April 2024, 06:36:50 UTC
df0fd30 [Doc] Privileges required to retrieve the status of async searches Document that users can retrieve the status of the async searches they submitted without any extra privileges. 02 April 2024, 06:35:02 UTC
4ddb640 Remove o.e.painless.toxcontent.UserTreeToXContent (#106935) This was never used -> remove it and its test. 02 April 2024, 00:41:22 UTC
393e644 Refactor more ActionListener.wrap to delegateFailureAndWrap (#106948) Refactoring a couple more of these and inlining some listeners while at it to save code/memory and get cleaner guarantees around note leaking things. Note: In two cases inlining makes it obvious that we can remove spurious AtomicReference usage. 01 April 2024, 22:44:44 UTC
22ca5ec Fix warning typo for test failure (#106971) This had the wrong index patterns, which led to test failures Resolves #106965 01 April 2024, 22:19:59 UTC
6011763 Call out `monitor` privilege for index and component templates (#106970) These can be retrieved using the `monitor` cluster privilege. This just adds that note to the docs. 01 April 2024, 21:52:27 UTC
5cc4a56 AwaitsFix #106939 01 April 2024, 21:09:52 UTC
9077c5d AwaitsFix #106968 01 April 2024, 21:04:56 UTC
f9d96ae AwaitsFix #106900 01 April 2024, 20:44:31 UTC
8fdd1aa Refactor geoip database properties (#106960) 01 April 2024, 20:34:15 UTC
3eeffeb AwaitsFix #106964 01 April 2024, 20:33:22 UTC
a3794e7 [DOCS] Remove orphaned cluster issues troubleshooing doc (#106959) 01 April 2024, 20:12:48 UTC
3810a91 Block readiness on file settings being applied (#106437) The readiness service is supposed to wait on the cluster being formed as well as file settings being applied. Yet file settings application is only checked on the master node. Since master election itself does not guarantee file settings have been applied, non-master nodes may mark themselves as ready even though they should not yet accept requests. This commit reworks how the readiness service looks ofr file settings being applied. Previously it only worked on the master node where it got a callback directly from the file settings service. With this change we now only look at cluster state. Note that this means the readiness service and file based settings are tightly coupled. In practice this was always the case as they are both meant to be used in cloud environments, but the coupling is a bit tighter now since the readiness service will never report ready unless file based settings exist. 01 April 2024, 19:49:29 UTC
8982513 Adjust array resizing in block builder (#106934) I looked into an async profiler and found that AbstractBlockBuilder#updatePosition was consuming a significant amount of CPU. This is because we're growing the firstValueIndexes array one by one. While this minimizes wasted memory, it requires more CPU. I think we should use ArrayUtil.oversize() to resize this array. In contrast, it appears that we're growing the values array too quickly by 50% each time. I think we should use ArrayUtil.oversize() with a growth rate of 1/8 here as well. 01 April 2024, 17:28:59 UTC
af93900 Remove Singleton annotation and downstream code from o.e.common.inject (#106936) We don't use the singleton annotation anywhere so it and a bunch of the scoping code can just go away. Also, we don't use the stage functionality (it's always develop) so that can go away as well. 01 April 2024, 15:53:51 UTC
25d3f70 ESQL: More tests for DATE_TRUNC (#106908) This adds integration tests for `DATE_TRUNC` that round to `1 HOUR` and `1 MINUTE` - that's a thing folks will do and I didn't see it in the integration tests. We do it in unit tests but I just want to be extra paranoid. 01 April 2024, 15:37:01 UTC
af8de04 Remove redundant testing of Java 22 (#106943) 01 April 2024, 15:23:27 UTC
f078c30 AwaitsFix #106957 01 April 2024, 15:17:26 UTC
f4613d0 ESQL: perform a reduction on the data node (#106516) * Introduce node-level reduction (instead of the coordinator level one) behind a pragma 01 April 2024, 14:33:03 UTC
89bf4b3 Make int8_hnsw our default index for new dense-vector fields (#106836) For float32, there is no compelling reason to use all the memory required by default for HNSW. Using `int8_hnsw` provides a much saner default when it comes to cost vs relevancy. So, on all new indices that use `dense_vector` and want to index them for fast search, we will default to `int8_hnsw`. Users can still customize their parameters, or prefer `hnsw` over float32 if they so desire. 01 April 2024, 12:23:32 UTC
8644d3d Remove unused o.e.i.m.extras.RankFeatureQueryBuilders (#106951) This is unused and the last commit referencing it was in 21. 01 April 2024, 12:23:01 UTC
de6ece6 Check that operations on the Kibana system index use system index thread pools (#106915) * Add test to verify Kibana's access to system thread pools * Block all relevant threads on any number of nodes * Fill threadpool queues before testing requests that should be blocked 01 April 2024, 00:57:25 UTC
b643aba Harden usage of XML document builder in build-conventions (#106874) While `LicenseHeadersTask` only uses `DocumentBuilderFactory` internally to parse `rat.xml` files (which are the output of running Apache RAT on Elasticsearch codebase files), it is a good practice to disable XXE features even if it's part of checks that are run on developers machines. 31 March 2024, 23:41:52 UTC
64847b3 AwaitsFix for #104081 31 March 2024, 20:51:38 UTC
60d5083 Remove some more ActionType subclasses (#106947) Cleaning up a couple more of these from the server module. 31 March 2024, 19:49:30 UTC
b676d38 Update mixed cluster test skip version for downsampling (#106942) All nodes on the mixed cluster need to be at least on version 8.10 since PR #97557 introduced execution of downsampling tasks using the persisten task framework which is incompatible with how execution was coordinated before. 30 March 2024, 23:17:20 UTC
ddd7c53 Remove executor lookups from `TransportInstanceSingleOperationAction` (#106940) Replaces the `String` names (plus associated threadpool) with proper `Executor` instances. Relates #106279 (removes another usage of `SAME`) Relates #106938, #105460, #99787, #97879 etc. 30 March 2024, 19:33:22 UTC
7f83189 Remove executor lookups from `TransportWriteAction` (#106938) Replaces the `String` names (plus associated threadpool) with proper `Executor` instances. Relates #106279 (removes a couple more usages of `SAME`) Relates #105460, #99787, #97879 etc. 30 March 2024, 19:23:27 UTC
7af3c8d AwaitsFix #106871 30 March 2024, 18:13:05 UTC
d125f52 AwaitsFix #106933 30 March 2024, 01:36:05 UTC
ee2733f Fix MRJAR test tasks when runtime java is set (#106929) The MRJAR test tasks explicitly set the java toolchain to use for execution. This is incompatible with setting the test executable directly, which is what happens when runtime java is set. This commit guards setting the toolchain to only occur when runtime java isn't set, and adds an onlyIf to ensure the test is runnable when runtime java is set. relates #106828 29 March 2024, 21:57:02 UTC
c6a0d4f Pulling KeyValueProcessor.logAndBuildException() into AbstractProcessor (#106931) 29 March 2024, 21:29:16 UTC
4923f96 Tidy up the geoip Property enum (#106930) 29 March 2024, 21:22:31 UTC
f0fa715 Do not retain whitelist statically in painless plugin (#106913) We only need this single item list as well as the whitelist map once when setting up the script engine. Not holding on to them statically saves about 1.4M of heap. => don't hold on to the list statically => null out the map that we need across method calls (and fix a potential concurrency issue with the existing solution ...). 29 March 2024, 21:21:54 UTC
6b419c1 TransportNodesAction supports async node response (#106733) Gives children classes of TransportNodesAction the option to access to the request listener so as to respond asynchronously to incoming node requests. Relates ES-6685 29 March 2024, 19:53:16 UTC
back to top