https://github.com/elastic/elasticsearch

sort by:
Revision Author Date Message Commit Date
124eef1 Merge remote-tracking branch 'origin/main' into lucene_snapshot 25 August 2024, 10:02:12 UTC
7fc218f [Automated] Update Lucene snapshot to 9.12.0-snapshot-83df2ff10e2 25 August 2024, 06:11:26 UTC
48dabe8 Mute org.elasticsearch.smoketest.WatcherYamlRestIT test {p0=watcher/usage/10_basic/Test watcher usage stats output} #112189 25 August 2024, 04:17:01 UTC
d053d39 Mute org.elasticsearch.xpack.esql.qa.mixed.MixedClusterEsqlSpecIT test {mv_percentile.FromIndex ASYNC} #112188 24 August 2024, 20:47:15 UTC
ecd7930 Mute org.elasticsearch.xpack.esql.qa.mixed.MixedClusterEsqlSpecIT test {mv_percentile.FromIndex SYNC} #112187 24 August 2024, 20:47:05 UTC
22ca381 Make slow logger's Logger instances static (#112183) These can be made static now that they aren't index specific any longer, saving measurable time in test execution + it's just the right thing to do here. 24 August 2024, 13:17:11 UTC
4c200f6 Make ClusterSettings.BUILT_IN_CLUSTER_SETTINGS immutable (#112185) This should be a final and immutable field. Also the way this thing was constructed is weird, we don't expect any null values for constants. 24 August 2024, 13:09:50 UTC
7628e10 Merge remote-tracking branch 'origin/main' into lucene_snapshot 24 August 2024, 10:03:29 UTC
0fa89d1 [Automated] Update Lucene snapshot to 9.12.0-snapshot-83df2ff10e2 24 August 2024, 06:12:39 UTC
1502352 Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {mv_percentile.FromIndex SYNC} #112180 24 August 2024, 05:59:34 UTC
2c94068 Allow query pragmas in release builds (#111953) I have investigated an issue with QA clusters that run release builds. I wish I could enable query pragmas to confirm the problem instead of setting up new clusters and replicating data before testing the theory. This change allows users to enable query pragmas in release builds. However, due to the risks associated with using pragmas, the accept_pragma_risks parameter must be explicitly set to true to proceed. 23 August 2024, 20:30:28 UTC
9d6bef1 Docs: Scripted metric not available in serverless (#112161) This updates the docs to say that scripted metric is not available in serverless. 23 August 2024, 19:26:46 UTC
8c85d44 ESQL: Lock some data types in profile test (#112165) The test for the output from `profile` can sometimes return `long` and sometimes return `int`. That's fine, really. It just makes testing annoying. This promotes the types to always be a `long` in the test. Closes #112049 Closes #112039 23 August 2024, 19:23:25 UTC
0f6529d Improve threading when restoring snapshot (#112162) We use the `SNAPSHOT_META` pool for some of the work needed to start a snapshot restore, but it's a little tangled with other work happening on a `transport_worker`, or some other random threadpool on which `getRepositoryData` completes. This commit ensures that we use `SNAPSHOT_META` throughout. Relates #101445 23 August 2024, 19:06:27 UTC
75b0c5d Ignore under construction data types for params checking (#112163) This should prevent tests failing when we've added a type to the params annotation but the data type is disabled via feature flag. 23 August 2024, 19:00:05 UTC
0aa4758 Stop setting java.library.path (#112119) Native libraries in Java are loaded by calling System.loadLibrary. This method inspects paths in the java.library.path to find the requested library. Elasticsearch previously used this to find libsystemd, but now the only remaining use is to set the additional platform directory in which Elasticsearch keeps its own native libraries. One issue with setting java.library.path is that its not set for the cli process, which makes loading the native library infrastructure from clis difficult. This commit reworks how Elasticsearch native libraries are found in order to avoid needing to set java.library.path. There are two cases. The simplest is production, where the working directory is the Elasticsearch installation directory, so the platform specific directory can be constructed. The second case is for tests where we don't have an installtion. We already pass in java.library.path there, so this change renames the system property to be a test specific property that the new loading infrastructure looks for. 23 August 2024, 18:16:18 UTC
1fb2afa Re-enable yaml tests (#112157) Related to #112143 23 August 2024, 16:26:57 UTC
d716541 [DOCS] Wrap document/field restriction tip in IMPORTANT block (#112146) 23 August 2024, 16:23:57 UTC
3c92797 Allow warnings for template conflicts (#112145) Fixes #112143 23 August 2024, 14:53:07 UTC
9847a31 Semantic reranking should fail whenever inference ID does not exist (#112038) * Semantic reranking should fail whenever inference ID does not exist * Short circuit text similarity reranking on empty result set * Update tests * Remove test - it doesn't do anything useful * Update docs/changelog/112038.yaml 23 August 2024, 14:41:03 UTC
bcad4f0 Mute org.elasticsearch.smoketest.SmokeTestMultiNodeClientYamlTestSuiteIT org.elasticsearch.smoketest.SmokeTestMultiNodeClientYamlTestSuiteIT #112147 23 August 2024, 13:59:44 UTC
915528c Mute org.elasticsearch.xpack.test.rest.XPackRestIT test {p0=transform/preview_transforms/Test preview transform latest} #112144 23 August 2024, 13:08:09 UTC
3ab1634 Mute org.elasticsearch.test.rest.ClientYamlTestSuiteIT org.elasticsearch.test.rest.ClientYamlTestSuiteIT #112143 23 August 2024, 12:32:20 UTC
db0cc81 Add support for spatial relationships in point field mapper (#112126) Lucene only supports intersects queries over XYPoint fields but it is still possible to represent all the spatial relationships using just that query. 23 August 2024, 12:02:14 UTC
2b11705 Change subobjects yaml tests to use composable index templates. (#112129) Currently the legacy templates are being used which are deprecated. 23 August 2024, 10:17:54 UTC
d6d3058 Fix synthetic source NestedObjectMapper assertion. (#112131) The parentDoc parameter can be -1 and the assertion needs to take this into account (just like the next line is doing). Closes #111998 23 August 2024, 10:17:05 UTC
7796473 Merge remote-tracking branch 'origin/main' into lucene_snapshot 23 August 2024, 10:01:59 UTC
34a78f3 Add documentation to deprecate the global retention privileges. (#112020) 23 August 2024, 08:49:15 UTC
92d25c1 Fix id and routing types in indices.split YAML tests (#112059) 23 August 2024, 08:46:52 UTC
8325a71 Use StandardAnalyzer in MapperServiceTestCase (#112127) We currently use Lucene's `MockAnalyzer` that rarely injects some random payload to text fields. This leads to assert errors for synthetic source, where the roundtrip source (after printing and parsing the synthetic source) appears the same but there's a difference now in the FieldInfo for text mappers due to the injected payload. Fixes #112083 23 August 2024, 08:11:47 UTC
e46b517 Minor cleanup of code in the org.elasticsearch.index.codec package. (#112125) * Removing unnesesary field * making inner class static * use enhaunced switch statement * removed commented out code. * made immutable fields final 23 August 2024, 07:59:38 UTC
e0c1ccb Make enrich cache based on memory usage (#111412) The max enrich cache size setting now also supports an absolute max size in bytes (of used heap space) and a percentage of the max heap space, next to the existing flat document count. The default is 1% of the max heap space. This should prevent issues where the enrich cache takes up a lot of memory when there are large documents in the cache. 23 August 2024, 07:26:55 UTC
e1dc596 SLM interval schedule followup - add back getFieldName style getters (#112123) Recent SLM interval change #110847 included changing two classes to records. This changed the getter methods from the form getFieldName() to fieldName(). Unfortunately, serverless expected the fieldName() form. Until serverless can be updated, we'll add back the getFieldName() style getters, in addition to the fieldName() getters, so as not to break the build. 23 August 2024, 07:21:11 UTC
a3ffb12 [Automated] Update Lucene snapshot to 9.12.0-snapshot-b5c410da870 23 August 2024, 06:12:18 UTC
1072f2b Add interval based SLM scheduling (#110847) Add the ability to schedule an SLM policies with a time unit interval schedule rather than a cron job schedule. For example, an slm policy can be created with the argument "schedule":"30m". This will create a policy that will run 30 minutes after the policy modification_date. It will then run again every time another 30 minutes has passed. Every time the policy is changed, the next snapshot will be re-scheduled to run one interval after the new modification date. 23 August 2024, 02:15:29 UTC
14b7170 Don't fail retention lease sync actions due to capacity constraints (#109414) Closes #105926 22 August 2024, 23:13:41 UTC
7b1d2a2 Mute org.elasticsearch.xpack.esql.qa.mixed.MixedClusterEsqlSpecIT test {stats.ByTwoCalculatedSecondOverwritesReferencingFirst SYNC} #112118 22 August 2024, 20:44:31 UTC
5dcdc34 Mute org.elasticsearch.xpack.esql.qa.mixed.MixedClusterEsqlSpecIT test {stats.ByTwoCalculatedSecondOverwrites SYNC} #112117 22 August 2024, 20:43:13 UTC
d802e6f Test infra: Catch and report errors building error (#112109) This modifies our `matchesMap` and `matchesList` infrastructure to report when it encounters an error building the description of an error. This looks something like: ``` a map containing foo: expected "val" but error describing <java.lang.IllegalStateException: intentional failure ... a whole stack trace ...> bar: <val> ``` This preserves the original error message while also giving you the context for the actual failure. Relates to #112039 Relates to #112049 22 August 2024, 20:04:32 UTC
16a12ff Revert "[plugin/apm-data] Set fallback to legacy ILM policies" (#112112) * Revert "[plugin/apm-data] Set fallback to legacy ILM policies (#112028)" This reverts commit fd37ef88c28744181d4628a05baed57098884bd9. 22 August 2024, 18:03:32 UTC
5534a1f Fix Test Failure in SplitIndexIT (#112070) This PR fixes the testSplitIndexPrimaryTerm() test inside SplitIndexIT. Fixes #111282 22 August 2024, 16:34:45 UTC
322d319 Mute org.elasticsearch.xpack.esql.EsqlAsyncSecurityIT testLimitedPrivilege #112110 22 August 2024, 15:39:25 UTC
0f176e1 Remove leftover libsystemd references (#112078) Systemd notification now happens by directly communicating with the systemd socket. This commit removes the native access to libsystemd, which is no longer used. 22 August 2024, 14:57:15 UTC
ed60470 Display effective retention in the relevant data stream APIs (#112019) 22 August 2024, 14:42:49 UTC
6d076df ESQL: fix for missing indices error message (#111797) Reverts a part of https://github.com/elastic/elasticsearch/pull/109483 by going back to the previous (more restrictive) way of dealing with missing indices or aliases. More specifically, if an index pattern used in a query refers to a missing index or alias name and doesn't use a wildcard for this name, then we error out. Our lack of testing in this area made the change in https://github.com/elastic/elasticsearch/pull/109483 to be invisible. Fixes https://github.com/elastic/elasticsearch/issues/111712 22 August 2024, 14:20:47 UTC
f37440f (Doc+) Allocation Explain Examples: THROTTLED, MAX_RETRY (#111558) Adds [Allocation Explain examples](https://www.elastic.co/guide/en/elasticsearch/reference/master/cluster-allocation-explain.html#cluster-allocation-explain-api-examples) for `THROTTLED` and `MAX_RETRY`. Also formats sub TOC so that we can after link code message to those docs. 22 August 2024, 14:16:36 UTC
de14c1d Fix testSuggestProfilesWithHint (#112010) 22 August 2024, 14:08:31 UTC
615e084 Add more cross-links about sniff/proxy modes (#112079) The info about remote cluster connection modes is a little disjointed. This commit adds some cross-links between the sections to help users find more relevant information. 22 August 2024, 13:13:56 UTC
f481b07 Always check crsType when folding spatial functions (#112090) * Always check crsType when folding spatial functions * Update docs/changelog/112090.yaml * Only require capability for fixed test The other tests passed on older versions anyway. 22 August 2024, 12:51:19 UTC
c1daf18 ESQL: Support INLINESTATS grouped on expressions (#111690) This adds support for grouping `INLINESTATS` on an expression: ``` | INLINESTATS MAX(avg_worked_seconds) BY SUBSTRING(last_name, 0, 1) ``` This functions *exactly* as thought you ran did: ``` | EVAL `SUBSTRING(last_name, 0, 1)` = SUBSTRING(last_name, 0, 1) | INLINESTATS MAX(avg_worked_seconds) BY `SUBSTRING(last_name, 0, 1)` ``` The calculated field is retained in the results. This works by running the `LogicalPlanOptimizer` before forking off plan phases. If we get sub-phases then we rerun the `LogicalPlanOptimizer` on each phase so we can fuse *stuff*. Then I had modify the optimizer rule that implements expressions in the `BY` position on `STATS` so it worked on `INLINESTATS`. And that's it? That's it?! Really? 22 August 2024, 12:40:35 UTC
1362d56 Introduce mode `subobjects=auto` for objects (#110524) * Introduce mode `subobjects=auto` for objects * Update docs/changelog/110524.yaml * compilation error * tests and fixes * refactor * spotless * more tests * fix nested objects * fix test * update fetch test * add QA coverage * update tests * update tests * update tests * fix nested 22 August 2024, 12:13:52 UTC
f5c250b Fix test compilation for new Lucene 912 Codec (#112093) This commit fixes a compilation issue by updating the test to use the new Lucene 912 codec. 22 August 2024, 11:25:11 UTC
585bd64 Add H3 Benchmarks (#111359) Microbenchmarks for H3 22 August 2024, 10:56:31 UTC
62305f0 Updates-warning-about-mounting-snapshots (#112057) * Updates-warning-about-mounting-snapshots * Update docs/reference/searchable-snapshots/apis/mount-snapshot.asciidoc Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com> --------- Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com> 22 August 2024, 10:22:32 UTC
fb6c5a5 Update Gradle wrapper to 8.10 (#111736) 22 August 2024, 10:15:22 UTC
0c8f5cc Merge remote-tracking branch 'origin/main' into lucene_snapshot 22 August 2024, 10:02:11 UTC
31cdc43 Mute org.elasticsearch.xpack.ml.integration.MlJobIT testDeleteJobAfterMissingIndex #112088 22 August 2024, 08:08:41 UTC
967af10 Fix DocValuesCodecDuelTests testDuel (#112084) We need to check the returned doc id from advance() before accessing the values of the current document. Closes #112082 22 August 2024, 07:08:38 UTC
fe786b7 Force implementing bulk InputStream#read on StreamInput (#112072) We should enforce overriding here to avoid extremely slow byte-by-byte reads when using these instances as `InputStream`. I only found one case where this matters practically in the codebase but it's probably good to guard against it. 22 August 2024, 07:07:42 UTC
4b30342 [Automated] Update Lucene snapshot to 9.12.0-snapshot-fc011d36e7d 22 August 2024, 06:12:02 UTC
10d665b Collect APM metrics for failure stores (#108279) This PR adds APM metrics for failure stores. See the JavaDoc comments in `FailureStoreMetrics.java` for a detailed explanation on the individual metrics. 22 August 2024, 06:10:12 UTC
1589015 Mute org.elasticsearch.xpack.inference.InferenceRestIT test {p0=inference/80_random_rerank_retriever/Random rerank retriever predictably shuffles results} #111999 22 August 2024, 05:09:35 UTC
a226826 Mute org.elasticsearch.index.codec.tsdb.DocValuesCodecDuelTests testDuel #112082 22 August 2024, 04:28:36 UTC
7759b55 Improve zstd test coverage by adding a test duel (#112048) Adding test duel between out of the box stored fields codec and zstd stored field codecs: - lz4 compared to zstd level 0 (best speed) - deflate compared zstd level 3 (best compression) Relates #108706 22 August 2024, 02:31:51 UTC
fd37ef8 [plugin/apm-data] Set fallback to legacy ILM policies (#112028) 21 August 2024, 23:12:24 UTC
f0dbda7 Expand docs on remote cluster proxying (#112025) It's not obvious from the docs that transport connections (including connections to remote clusters) use a custom binary protocol and require a _layer 4_ proxy. This commit clarifies this point. 21 August 2024, 21:26:57 UTC
6dd3107 ESQL: Speed up attribute serialization tests (#112069) Speeds up the attribute serialization tests by building the random configuration one time, rather than over and over and over again. It's expensive to make a random configuration! 21 August 2024, 20:18:42 UTC
a35da24 Merge multiple ignored source entires for the same field (#111994) 21 August 2024, 18:21:41 UTC
a281bf8 Fix RRF validation for rank_constant < 1 (#112058) * Fix RRF validation for t push rank_constant < 1 * Add yaml test * Update docs/changelog/112058.yaml 21 August 2024, 16:59:06 UTC
bf1ec5d ESQL: Add async ID and is_running headers to ESQL async query (#111840) Add headers to async ESQL queries to show the status and query ID without having to parse the body. ESQL part of #109576 21 August 2024, 15:47:41 UTC
d76e1af Fix calculation of parent offset for ignored source in some cases (#112046) 21 August 2024, 15:46:40 UTC
27721c3 Add a test reproducing issue with lookup of parent document in nested field synthetic source (#112043) 21 August 2024, 15:36:55 UTC
d38aeab ESQL: Drop cast in aggs (#112035) Drops all `Math.toIntExact` calls in aggs that which aren't needed because groups *are* `int` already. 21 August 2024, 15:23:47 UTC
fe75aa3 Enable LTR in serveless. (#111102) 21 August 2024, 15:21:16 UTC
e9b2e57 Unskip mv_percentile tests and add extra debugging data (#112061) Closes https://github.com/elastic/elasticsearch/issues/112036 Closes https://github.com/elastic/elasticsearch/issues/112037 This isn't fixing anything, as I couldn't reproduce the issue. But at least now, if it fails again, we know which data rendered that result exactly (Or so I wish...) 21 August 2024, 14:48:51 UTC
ba87a48 Handle BigInteger in xcontent copy (#111937) When xcontent is copied, the parse tree is walked and each element is passed to the given generator. In the case of numbers, BigInteger is currently not handled. Although arbitrary precision BigIntegers are not supported in Elasticsearch, they appear in xcontent when using unsigned long fields. This commit adds handling for that case, and also ensures all token types are handled. Note that BigDecimal are not supported at all since double is the largest floating point mapper supported. closes #111812 21 August 2024, 14:12:02 UTC
84ddd6c [DOCS] Update `rank_constant` value in retriever example (#112056) 21 August 2024, 13:11:19 UTC
fa7f836 Update searchable snapshot doc about the timing to notice data loss (#112050) <!-- Thank you for your interest in and contributing to Elasticsearch! There are a few simple things to check before submitting your pull request that can help with the review process. You should delete these items from your submission, but they are here to help bring them to your attention. --> - Have you signed the [contributor license agreement](https://www.elastic.co/contributor-agreement)? => yes - Have you followed the [contributor guidelines](https://github.com/elastic/elasticsearch/blob/main/CONTRIBUTING.md)? => yes - If submitting code, have you built your formula locally prior to submission with `gradle check`? => not code - If submitting code, is your pull request against main? Unless there is a good reason otherwise, we prefer pull requests against main and will backport as needed. => not code - If submitting code, have you checked that your submission is for an [OS and architecture that we support](https://www.elastic.co/support/matrix#show_os)? => not code - If you are submitting this code for a class then read our [policy](https://github.com/elastic/elasticsearch/blob/main/CONTRIBUTING.md#contributing-as-part-of-a-class) for that. => not code ## Description Update searchable snapshot doc about the timing to notice data loss: Sometimes searchable snapshot data is cached onto disk so user may notice their data loss later during node restart (or on Elastic cloud - host maintenance) after they delete their snapshots. 21 August 2024, 12:48:06 UTC
2b7a3cd Unmuted ModelRegistry testGetModel (#112042) 21 August 2024, 12:37:22 UTC
277df6d Merge remote-tracking branch 'origin/main' into lucene_snapshot 21 August 2024, 10:01:58 UTC
f5de9c0 (Doc+) "min_primary_shard_size" for 10-50GB shards (#111574) 👋🏽 howdy, team! Expands [10-50GB sharding recommendation](https://www.elastic.co/guide/en/elasticsearch/reference/master/size-your-shards.html#shard-size-recommendation) to include ILM's more recent [`min_primary_shard_size`](https://www.elastic.co/guide/en/elasticsearch/reference/master/ilm-rollover.html) option to avoid small shards. 21 August 2024, 09:57:09 UTC
5d5d2e8 [ESQL] Fix cases of geometry collections with one point in CartesianPoint queries (#111193) 21 August 2024, 09:39:54 UTC
2681bb8 ESQL: Added exception catching to aggregators (#111829) Add the `warnExceptions` capability to aggregators. Same as for evaluators. This requires: - Having a "Warnings-like" object in the compute module. I created a new class for that, temporarily - Added new "FaillibleState" objects, that hold a "failed" boolean/bitset to mark failed groups - Catching exceptions in all combine methods receiving a single group (Not catching in evaluateFinal/Intermediate, nor in combines that receive the full state) - Shortcircuiting operations if the group is in "failed" state - Having a third intermediate state: `values, seen, failed` Extracted from https://github.com/elastic/elasticsearch/pull/111639. Check it to see a use case 21 August 2024, 08:18:21 UTC
0d38528 Fix testLicenseTombstoneWithUsedTrialFromXContext (#112051) Relates: https://github.com/elastic/elasticsearch/issues/103093 21 August 2024, 07:11:48 UTC
a121fcb Fix validation expception message if adaptive allocations is behind feature flag (#111970) 21 August 2024, 06:41:07 UTC
a294265 Re-enable xpack yaml tests (#112031) 21 August 2024, 06:34:08 UTC
5b7d985 [Automated] Update Lucene snapshot to 9.12.0-snapshot-25253a1a016 21 August 2024, 06:11:22 UTC
3153bd0 Mute org.elasticsearch.xpack.esql.qa.single_node.RestEsqlIT testForceSleepsProfile {ASYNC} #112049 21 August 2024, 04:32:43 UTC
3b8c9ad Pin default codec to index mode. (#111997) By default, the 'default' codec will be used as default, in case of logsdb index mode best_compression codec will be used as default. 21 August 2024, 02:51:26 UTC
29e922c Remove search worker pool (#111099) No more need for this pool, now that Lucene can safely execute on the current pool. 20 August 2024, 21:57:38 UTC
c1019d4 (Doc+) Link API doc to parent object - part1 (#111951) * (Doc+) Link API to parent Doc part1 --------- Co-authored-by: shainaraskas <shaina.raskas@elastic.co> Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com> 20 August 2024, 20:58:18 UTC
3a122a7 Mute org.elasticsearch.xpack.esql.qa.single_node.RestEsqlIT testForceSleepsProfile {SYNC} #112039 20 August 2024, 20:24:58 UTC
29f9ad9 Allow for negative epoch seconds (#111938) The change to allow nanoseconds in ZonedDateTime split the epoch seconds from the nanosecond subelement. However, the epoch seconds were then written as a vlong, when in fact they could be negative if the date is before epoch. This commit changes the format to use zlong instead, which supports negatives. closes #111923 20 August 2024, 19:45:20 UTC
94aa6ce Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {mv_percentile.FromIndex ASYNC} #112037 20 August 2024, 19:16:33 UTC
fe5fa05 Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {mv_percentile.FromIndex SYNC} #112036 20 August 2024, 19:16:20 UTC
8841c61 EsqlCapabilities for casting string to version (#112032) 20 August 2024, 18:44:56 UTC
94c48ac [ML] Remove HttpClientContext (#111914) No requests are making use of this parameter, and it is clashing with Bedrock. We will bring it back in a later change as part of the Http Request portion of the Sender. 20 August 2024, 18:35:39 UTC
0f8ce78 [ML] Migrate Inference to ChunkedToXContent (#111655) In preperation of streaming Inference responses, we are migrating RestInferenceAction and corresponding result objects from ToXContent to ChunkedToXContent. RestInferenceAction will now use the built in ChunkedRestResponseBodyPart and send a single item before closing the stream. 20 August 2024, 18:34:44 UTC
a9fa443 Avoid unsafe futures in SharedBlobCacheService (#111526) 20 August 2024, 17:59:35 UTC
f2021f4 Unmute the test, it had already been fixed but wasn't unmuted (#111800) 20 August 2024, 16:52:30 UTC
back to top