https://github.com/mperham/sidekiq

sort by:
Revision Author Date Message Commit Date
873f1df changes, release 02 April 2018, 16:04:41 UTC
bca9396 Fix Ruby version comparison This line previously failed against 2.2.10. 29 March 2018, 20:07:14 UTC
6614abd release 29 March 2018, 19:08:40 UTC
c262f76 release 29 March 2018, 19:04:02 UTC
b23c0ec Use `RAILS_MAX_THREADS` for client pool size This is a follow up to #2985 (52828e4) adding similar support for the client connection pool. For Rails servers, Sidekiq is not loaded from the CLI so the prior change to support setting the concurrency via `RAILS_MAX_THREADS` is not applied to the web server process. This means for Rails servers which do not configure a custom size through an initializer they will run with the default connection pool size of 5. When the Rails server runs the initial Redis connection may be made through `Sidekiq::Client` (e.g. from [`ActiveJob::QueueAdapters::SidekiqAdapter`](https://github.com/rails/rails/blob/v5.1.5/activejob/lib/active_job/queue_adapters/sidekiq_adapter.rb#L20)). This causes the `redis_pool` to be initialized without any options, setting the pool size to the default of 5. .gem/ruby/2.5.0/gems/sidekiq-5.1.1/lib/sidekiq.rb:125:in `redis_pool' .gem/ruby/2.5.0/gems/sidekiq-5.1.1/lib/sidekiq/client.rb:42:in `initialize' .gem/ruby/2.5.0/gems/sidekiq-5.1.1/lib/sidekiq/client.rb:131:in `new' .gem/ruby/2.5.0/gems/sidekiq-5.1.1/lib/sidekiq/client.rb:131:in `push' .gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/queue_adapters/sidekiq_adapter.rb:20:in `enqueue' .gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/enqueuing.rb:51:in `block in enqueue' .gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:108:in `block in run_callbacks' .gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/logging.rb:15:in `block (3 levels) in <module:Logging>' .gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/logging.rb:44:in `block in tag_logger' .gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/tagged_logging.rb:69:in `block in tagged' .gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/tagged_logging.rb:26:in `tagged' .gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/tagged_logging.rb:69:in `tagged' .gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/logging.rb:44:in `tag_logger' .gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/logging.rb:14:in `block (2 levels) in <module:Logging>' .gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:117:in `instance_exec' .gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:117:in `block in run_callbacks' .gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:135:in `run_callbacks' .gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/enqueuing.rb:47:in `enqueue' .gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/enqueuing.rb:18:in `perform_later' For the majority of cases, a client pool size of 5 is sufficient. However, servers which utilize a high number of threads, with large job payloads, and which may experience some network latency issues can see `Timeout::Error` crashes. This may be further exacerbated by the ~2-20x performance decrease through `ActiveJob` (#3782). Rails addresses this general type of connection issue for the main database by suggesting that the DB pool size match the number of threads running. This change applies that logic to the default client pool size by leveraging the same environment setting; this way there's a connection available per thread. This may also have the side effect of a slight performance boost, as there is less of a chance that threads will be blocked waiting on connections. The trade-off is that there may be a memory profile increase to handle the additional Redis connections in the pool; note the pool only creates new connections as necessary to handle the requests. Resolves #3806 27 March 2018, 22:46:57 UTC
60a41d3 change 27 March 2018, 16:18:52 UTC
82bdae8 Add link to docs in footer 27 March 2018, 16:18:05 UTC
86ca02f Refactor, changes 22 March 2018, 16:13:21 UTC
7448351 fix: use default when returned 0 22 March 2018, 16:04:28 UTC
611f7cd Use default retry interval in sidekiq_retry_in 22 March 2018, 16:04:28 UTC
1aad7b5 fixes 19 March 2018, 04:59:45 UTC
967f3f3 Add missing preamble for @krzysiek1507, #3759 15 March 2018, 16:00:53 UTC
aa46700 Allow use of AR middleware outside of Rails proper, fixes #3787 13 March 2018, 16:00:50 UTC
8a4926c Modern optimization 27 February 2018, 20:50:33 UTC
a0bcd59 changes, bump 26 February 2018, 18:05:59 UTC
6b3e453 Issue 3775 - Fixed Ctrl-C exception on Windows. 26 February 2018, 17:59:05 UTC
e18d9ac Fix pro version link 22 February 2018, 22:26:49 UTC
c2a7ee7 change 22 February 2018, 00:40:27 UTC
ee3194d 401 21 February 2018, 19:07:11 UTC
2ed9260 Missed a few 17 February 2018, 22:09:28 UTC
7de6f4c Remove explicit freeze, which should not be necessary anymore with frozen_string_literal: true. Fixes #3759 17 February 2018, 22:09:28 UTC
abca42d Revert "Use active record cache when available" This reverts commit a27d692656e15636cebe74542371a5e93f3f9194. The reason for reverting is that this middleware has no effect. In Rails 5.0+, Sidekiq runs its jobs using the Rails reloader/executor API[1]. In Rails 5.0+, however, the ActiveRecord query cache registers its own hooks with the executor[2], and these hooks automatically enable the query cache for the duration of any work the executor performs[3]. As a result, the query cache is already on by default for any Sidekiq jobs run in Rails 5.0+, so it's not necessary to separately enable it in a middleware. [1] https://github.com/mperham/sidekiq/blob/eca6acc0ce201125a45a5af75fd08c6ce985df5a/lib/sidekiq/rails.rb#L33-L37 [2] https://github.com/rails/rails/blob/f8c00c130016b248d1d409f131356632dcc418c6/activerecord/lib/active_record/railtie.rb#L159-L163 [3] https://github.com/rails/rails/blob/f8c00c130016b248d1d409f131356632dcc418c6/activerecord/lib/active_record/query_cache.rb#L26-L41 15 February 2018, 18:02:52 UTC
eca6acc comment 13 February 2018, 20:01:19 UTC
cc8fed0 changes 08 February 2018, 18:23:42 UTC
abfbc1e Revert "Make links out of all issue numbers in the changelog" This reverts commit 4f0efea6722f7c044a901e27ebfbd593fac7fa33. 08 February 2018, 18:21:20 UTC
10b0ff9 Merge branch 'master' of github.com:mperham/sidekiq 07 February 2018, 16:58:37 UTC
3f13f01 allow redis 3 07 February 2018, 16:58:31 UTC
da0bdd8 Clean up examples 06 February 2018, 20:27:14 UTC
b4d124d bump 06 February 2018, 17:31:23 UTC
ad80323 Force Redis 4.x now, fixes #3748 06 February 2018, 16:58:55 UTC
77966a9 prep 03 February 2018, 23:40:10 UTC
329ea91 Merge branch 'master' of github.com:mperham/sidekiq 03 February 2018, 20:33:58 UTC
5fedc9b polish 03 February 2018, 20:33:26 UTC
4f0efea Make links out of all issue numbers in the changelog If your changelog says "see issue for details", you should make it easy for someone to get there. 03 February 2018, 00:12:52 UTC
b1523b9 3743 01 February 2018, 17:36:25 UTC
336532c Prepare for Pro 4 31 January 2018, 22:43:10 UTC
cdf6430 rename API to death handlers for consistency 31 January 2018, 18:22:22 UTC
87fc928 latest 31 January 2018, 17:39:51 UTC
b48ae5e Allow dynamic signal handlers 29 January 2018, 20:20:08 UTC
2df8e52 changes 26 January 2018, 20:23:11 UTC
e2244b1 Merge branch 'master' of github.com:mperham/sidekiq 26 January 2018, 20:18:01 UTC
5fc5966 Update default scheduler precision, fixes #3734 26 January 2018, 20:17:46 UTC
fbac0e5 Test against ruby 2.5 15 January 2018, 23:01:02 UTC
a2db45b Polish job failure notification 15 January 2018, 18:39:32 UTC
32d03ec Add global failure handlers, fixes #3721 12 January 2018, 22:13:45 UTC
dca9c56 bump 12 January 2018, 18:41:33 UTC
a27d692 Use active record cache when available This will prevent duplicated queries on database at the same job. 12 January 2018, 17:00:36 UTC
7cf7995 Fix japanese locale of web 12 January 2018, 15:40:22 UTC
54ffad5 changes 11 January 2018, 17:40:29 UTC
1e48865 Remove unused code 11 January 2018, 17:38:05 UTC
b9d4eae Exceptions raised during the startup event should kill the process, fixes #3717 11 January 2018, 17:37:55 UTC
89fcf0c Avoid r-n deprecation message, fixes #3716 11 January 2018, 15:12:29 UTC
e2a6227 ent changes 10 January 2018, 18:48:49 UTC
34e39aa changes 10 January 2018, 18:04:14 UTC
e72d51b Use of logger outside of Sidekiq server process, fixes #3714 10 January 2018, 18:04:09 UTC
b56afee Finally fix negative pending!?!?!? 10 January 2018, 17:55:11 UTC
ee89592 Make Sidekiq::Client.via reentrant, fixes #3715 Seems like an easy fix? 10 January 2018, 17:53:57 UTC
a9c9392 changes 20 December 2017, 23:08:10 UTC
ed9af50 constantize 20 December 2017, 18:47:48 UTC
071a235 load constants from within nested namespace rather than prioriting root namespace 20 December 2017, 14:47:53 UTC
84f17d9 Removal Of warninig message by not allowing to create singleton method if already created 18 December 2017, 15:41:54 UTC
b4c330b Sorting Of Available Locales 18 December 2017, 15:40:21 UTC
6ce47ef Add all changelogs to issue template 14 December 2017, 21:37:22 UTC
67f57eb 3.6 12 December 2017, 17:10:55 UTC
e447dae whoops, missed a few 08 December 2017, 21:26:50 UTC
5315e09 Ensure TID changes after fork, fixes #3685 08 December 2017, 21:24:21 UTC
515b294 changes for release 05 December 2017, 19:51:50 UTC
dcbcbbe bump 05 December 2017, 18:15:25 UTC
280c7f7 Symbolize all config.yml keys when loading This ensures that keys in the Yaml file can be specified as either symbols or strings, but will always be treated as symbols after loading. It tries to use ActiveSupport's `deep_symbolize_keys!` method if ActiveSupport is loaded, otherwise falling back to an inline version. Closes #3672. 02 December 2017, 21:37:32 UTC
4619888 remove unnecessary options, #3677 30 November 2017, 18:12:39 UTC
4a7da43 Log at error level on redis exceptions and then warn for exception Backout previous change to the exception handler signature and instead just log at error level the main part of the redis exception then just use the original exception handler as-is (at warn level) for the backtrace. This is a compromise in compatibility insofar as the backtrace is now at a lower log level, but alerting specifically on these error strings likely uses the main error strings explicitly logged before the very verbose backtrace. 30 November 2017, 18:11:16 UTC
aead218 Make error handler change fully backwards compatible The change that Fixes mperham/sidekiq#3673 actually broke any existing exception handlers due to differences in expected parameter count. This fixes it explicitly which normally seems like it won't matter but there are random monitoring gems that install their own sidekiq exception handlers that I don't want to break. 30 November 2017, 18:11:16 UTC
58d5ff1 Use exception handler for logging for fetch exception Extends the `ExceptionHandler` to support multiple log levels and additional context text in order to use it when the `Processor` encounters an exception when fetching new work. The added options ensure that the new behavior is fairly close to the old behavior (same log level, same fetch error message). Fixes mperham/sidekiq#3673 23 November 2017, 01:37:29 UTC
ae6f4aa no twitter support 18 November 2017, 19:44:29 UTC
80f217c bump jruby version to fix build? 15 November 2017, 14:33:57 UTC
2402bf3 :up: update redis namespace 15 November 2017, 14:15:44 UTC
bbf8c32 Fix message when require file could not be found sidekiq no longer supports rails 3. 12 November 2017, 20:49:02 UTC
5b03608 Change "really needs" to "must" in redis_connection.rb This is just a semantic change, but if the program is going to crash, I think "must have" is more prescriptive than "really needs to have". In my mind, "really needs to have" implies that the program will continue run but the user should do something differently in the future. 07 November 2017, 17:24:58 UTC
e19dd47 remove namespacea 02 November 2017, 16:03:37 UTC
63ee433 JobSet#each goes through elements in descending score order The current implementation of the #each method uses Redis.zrange to paginate the iteration and use multiple lightweight calls. It performs this pagination in descending score order, but each page is returned from Redis in ascending order. The result is that the final iteration through the whole set is not sorted properly. Here's an example with a page of size 3: Redis set: 1, 2, 3, 4, 5, 6, 7, 8, 9 JobSet.to_a: 7, 8, 9, 4, 5, 6, 1, 2, 3 This fixes it with barely no performance cost (each page is reverted in Ruby) and all the items are perfectly sorted in descending score order. 01 November 2017, 23:12:51 UTC
4dc90f0 Added warning to STDERR if we detect Rails 3.x (< 4) about Sidekiq not supporting this version any more 26 October 2017, 20:18:59 UTC
ee3490d Fix support link in README.md 26 October 2017, 16:26:02 UTC
28c53de Remove old testing/CI junk which hasn’t been used in years 20 October 2017, 16:09:03 UTC
b099bd1 new keys go to the bottom of the file 20 October 2017, 15:44:37 UTC
0ea2893 Add Latency key to en and es locales 20 October 2017, 15:43:11 UTC
8ff4484 Add latency row for each queue 20 October 2017, 15:43:11 UTC
25c0625 Merge branch 'master' of github.com:mperham/sidekiq 18 October 2017, 21:15:39 UTC
63f1f16 pro changes 18 October 2017, 21:15:34 UTC
fc85856 note customer support 10 October 2017, 16:41:14 UTC
90db3b8 release 02 October 2017, 19:53:14 UTC
167b871 Fix #3620 02 October 2017, 19:30:06 UTC
6332b9f redis-rb 4.0 support * Loosen Redis version dep to `>= 3.3.5, < 5` * Bump redis-namespace for looser Redis version dep. Pending https://github.com/resque/redis-namespace/pull/136 gem release. Use `redis.connection` where we can and fall back to `redis._client` where we need (to inspect timeout and scheme). References #3617 29 September 2017, 04:54:06 UTC
84273fb Update redis gem to avoid using the deprecated `client` API, fixes #3617 29 September 2017, 04:43:41 UTC
33b3bb1 pro and ent releases 14 September 2017, 19:44:22 UTC
36e75f5 pro changes 12 September 2017, 17:53:44 UTC
39532f8 Refactor Worker.set impl so it can be memoized, fixes #3602 12 September 2017, 17:49:05 UTC
de701c4 When a middleware mutates arguments, retry with original args Currently when a server middleware mutates arguments and then the job fails, it will get re-queued with the mutated arguments, not the original arguments. This is unexpected and (IMO) faulty behavior as that same middleware will now see the already-mutated arguments when the job gets executed again. This changes the behavior to use the pristine, unmutated jobs hash when requeueing the failed job. It also includes a failing test case for the old code. 11 August 2017, 23:33:55 UTC
d61a747 Add docs for the class arg of Client.push Document the difference between setting a string vs. a Sidekiq::Worker. 11 August 2017, 23:25:44 UTC
bd93f84 Unified Job kill via Sidekiq::DeadSet#kill 11 August 2017, 19:33:12 UTC
596c981 Show actual location, not :id, fixes #3560 08 August 2017, 17:43:27 UTC
back to top