https://github.com/wikimedia/operations-puppet

sort by:
Revision Author Date Message Commit Date
83fa138 StrictTransportSecurity for lists.wm.org this is a bit different because lists.wm.org uses lighttpd instead of Apache Bug: 38516 Change-Id: I53f27e7d82b720b5557f42ceef7d7bb26c95590c 15 August 2014, 03:02:48 UTC
0a466aa Merge "wikistats - use apache::site" into production 15 August 2014, 01:13:52 UTC
1601a59 fix lighttpd reload command Change-Id: Ib0c6dd6113c3eba0273bb7ec5325ca4e5622d7dd 14 August 2014, 23:21:44 UTC
8df528f fixup for lighttp being stupid about commas Change-Id: Icf16056feecb7ce0f33ef46df2296f65055f0617 14 August 2014, 23:12:42 UTC
f595c35 Make lists.wikimedia.org HTTPS only Bug: 68553 Change-Id: I3511f4b0d0185d1e4d35166c13f2104c7805f737 14 August 2014, 23:06:33 UTC
f5f6596 Bumped SlowQueryThreshold since the log is spammy and untruncated Change-Id: Ic6bc5a37be8f8668ad890e2a87a09d9366190031 14 August 2014, 22:03:42 UTC
b6ecd36 Disable cdh::hadoop::users temporarily Will fix this tomorrow. Change-Id: Iad874aba21407f570c1405a87a136abc786e1f07 14 August 2014, 21:03:33 UTC
2bb8528 Merge "svn - move Apache config from file to template" into production 14 August 2014, 21:00:44 UTC
cdffc00 Update cdh module with fix Change-Id: I0eb6e772e2d1bb2fe7de024246f637ab75fc8981 14 August 2014, 20:58:58 UTC
fea07f3 Merge "ganglia - use apache::site" into production 14 August 2014, 20:47:20 UTC
126894e Auto create HDFS user directories for users in a group This will make it easier for opsen to grant Hadoop access. They will no longer have to remember to also run these HDFS commands. Change-Id: Ia7bf6ec1254898807878a3415f663f159724bcb7 14 August 2014, 20:30:50 UTC
22afde1 Merge "performance.wm.org - use apache::site" into production 14 August 2014, 20:28:41 UTC
c44b064 add labmon1001 to site.pp RT: 7964 Change-Id: I95e87c0ffd94528b999b5da357fca57fb1d81d9d 14 August 2014, 19:27:24 UTC
9bcac99 Include role::analytics::refinery::data::check on analytics1027 Change-Id: I3e3eb955fe577f1207ed3d66436a51cc0aac5801 14 August 2014, 19:11:39 UTC
3f636cf add yuvipanda to labmon-roots admin group as requested in RT: 8027 (unless full root isn't needed and we can reduce it to running ALL as a certain user) Change-Id: I27f9faf8e4be96220126e910fbcc9fe3bd527158 14 August 2014, 19:09:20 UTC
07282fb introduce labmon-roots admin group add an admin group for full root access to labmon nodes Change-Id: I3a7b4c8ef048e1f15abc484cc5f4bd7f7fed4d44 RT: 8027 14 August 2014, 19:03:33 UTC
468d0ba Merge "ishmael - use apache::site" into production 14 August 2014, 18:50:02 UTC
1e8169d performance.wm.org - use apache::site use the newer apache::site method rather than a file resource to setup the Apache site Change-Id: I1abb57ef413d2b3b32c06636bb3d7df767b1a313 14 August 2014, 18:49:06 UTC
8a05a7e Merge "graphite - use apache::site" into production 14 August 2014, 18:41:38 UTC
d5667b7 Revert "wmflib: add ensure_final_newline()" This reverts commit 04945e956f835cb9849146db49df0559a6d28f94. Change-Id: Ida7f2e2c6b889cc2a04d3569bf8de5ea4e17ef2b 14 August 2014, 18:28:27 UTC
04945e9 wmflib: add ensure_final_newline() If the argument is a string, ensure it ends with a final newline. If it is not a string, pass it through unmodified. This function is designed to make it easier to write custom Puppet resource types that delegate to file resources and that take either a "content" or a "source" parameter. Change-Id: Ib1f88b11b6308900a0bf0279eaca863793f614cd 14 August 2014, 18:21:40 UTC
2195fe9 Merge "tendril - use apache::site" into production 14 August 2014, 18:19:31 UTC
5e30b0a ishmael - use apache::site use the newer apache::site method to setup Apache site instead of a file resource and have proper symlinks again current sitation is: neon:/etc/apache2/sites-enabled# file ishmael.wikimedia.org ishmael.wikimedia.org: ASCII text, with very long lines neon:/etc/apache2/sites-available# file ishmael.wikimedia.org ERROR: cannot open `ishmael.wikimedia.org' (No such file or directory) due to refactoring Change-Id: Iaf5879d55cde0ba87b718f6e340acf895fce5265 14 August 2014, 18:18:24 UTC
e14864f Override group permissions on /etc/send_nsca.cfg on Hadoop worker nodes This allows Hadoop processes to run send_nsca. Change-Id: I2529021517cb085dc526eacddb19231e290e0b14 14 August 2014, 18:04:34 UTC
eaa440f Increased the number of parsoid job runners to lower queue size Change-Id: If6f5d165e58883782669d1d79ffb9ffcd23e01d9 14 August 2014, 17:40:10 UTC
1dbd01d Revert "Remove mw1053 from mediawiki-installation dsh" This reverts commit 79b1c6e3e45bd59259f46d75c07463fc5870753e. Change-Id: I3e7a247008ac63ded33531aad53874aea6516981 14 August 2014, 16:01:46 UTC
7965a48 Additional python packages for stats servers * python-boto - Amazon S3 access (needed to get zero sms logs) * python-pandas - Pivot tables processing * python-requests - Simple lib to make API calls * python-unidecode - Unicode simplification - converts everything to latin set Change-Id: I271e081b6fe3503725ef03aba6607c7abfded383 14 August 2014, 14:58:38 UTC
34fedc3 Bumping up version of wikimetrics module Change-Id: I953ea5ee0ae270cdfba46e92955d94c7b83dabb1 14 August 2014, 14:06:31 UTC
8a44e5d Merge "Skip spam check if one of the recipients is postmaster@ or abuse@" into production 14 August 2014, 13:32:32 UTC
1051195 Merge "beta: fix ansi escapes for wmf-beta-autoupdater" into production 14 August 2014, 13:28:30 UTC
401e783 Merge "nutcracker: used ordered_yaml()" into production 14 August 2014, 12:54:29 UTC
07159b9 shell_exports(): sort keys to stabalize output The standard hash order thing. Change-Id: I77d82b7ba4cbf1efcbe2ae9c67ca2492b3391913 14 August 2014, 12:52:16 UTC
532a718 nutcracker: used ordered_yaml() ..rather than a hideous template. Change-Id: I4e878ca45038e261032611d373fd1690126bfa54 14 August 2014, 12:48:25 UTC
a1af57d Skip spam check if one of the recipients is postmaster@ or abuse@ Right now spam reports to abuse@ can be rejected at the SMTP layer if the spam score is too high. Let's avoid the issue altogether and not run SpamAssassin on these. Change-Id: I1ef3ab5606e4c47fb3e33c0554bc057c1277ff71 14 August 2014, 12:45:14 UTC
9a77edb Nutcracker: move declaration to role::mediawiki; parametrize * Set owner / group / mode for config files. * Only override nutcracker's default mbuf-size if caller requests it. * Don't declare defaults. Especially if they conflict with nutcracker's! * Don't assume a single server pool. Nutcracker supports arbitrarily many. We'll probably want to exploit this to ease the migration to a UNIX domain socket (if we choose to do that). * Move declaration to role::mediawiki / role::beta. * Fix Icinga alert to expect a single process. Change-Id: Ifb4d25506ad5dddf046c3e6ba03f30541850d2be 14 August 2014, 12:37:29 UTC
dc4b279 Merge "beta: Set runners_* for role::beta::jobrunner" into production 14 August 2014, 12:20:25 UTC
673fa31 mwgrep: Sort results before printing them Change-Id: I2d4d2dd3c3ffa8242ae65a76e83ebb6f63c57e7f 14 August 2014, 12:17:45 UTC
80865bb apache: change mpm dependency to work on trusty as well. Change-Id: Ibbfec0a1e07fd6f333b8121860f9e4b81bc9c4a5 Signed-off-by: Giuseppe Lavagetto <glavagetto@wikimedia.org> 14 August 2014, 10:56:39 UTC
9172721 jobrunner: use mpm_worker instead of mpm_prefork Change-Id: I9a1d31c7e81ed56d51267dcb98b856d7e38df768 Signed-off-by: Giuseppe Lavagetto <glavagetto@wikimedia.org> 14 August 2014, 10:52:02 UTC
09f2bea apache::conf: append trailing newline if missing If $content is not undefined, make sure it ends with a trailing newline. Change-Id: I863e2f9ed684c1d7e98c74c84db23d7ce066a404 14 August 2014, 10:31:55 UTC
f0306cd jobrunner: parametrize the port for the fcgi interface Change-Id: I8e92a0689c5ab41cc3fdbc8bb816bba356390ceb Signed-off-by: Giuseppe Lavagetto <glavagetto@wikimedia.org> 14 August 2014, 10:15:26 UTC
a75046b jobrunner: puppetize HAT runner The HAT jobrunner will spawn jobs by polling the fastcgi hhvm daemon via curl spawned by the jobrunner loop. In detail: * added a virtual host and the corresponding port definition * added the apache modules needed * disabled jit from command-line hhvm config Change-Id: I3d002968c30827b2e8c0fdf6fc55d8846080a104 Signed-off-by: Giuseppe Lavagetto <glavagetto@wikimedia.org> 14 August 2014, 09:04:31 UTC
558557e mariadb 10 config for sanitarium Change-Id: I2149f2e0b1ac5831a69e9cd7ef70e179b94fdaaa 14 August 2014, 05:11:53 UTC
89d1946 Unmanage ports.conf, following I0e4aa8800 Now that the file has been restored to its pristine state, it should not be managed by Puppet. Change-Id: Ib2dcd16c81ebdc8c3eeb91f9e01d3e06e3b57a9d 14 August 2014, 00:36:08 UTC
c01d744 puppetmaster - use apache::site use the newer apache::site method to setup the Apache site, rather than a file resource and have proper symlinks again currently, due to refactoring, it is like this: root@palladium:/etc/apache2/sites-available# file puppetmaster puppetmaster: ASCII English text root@palladium:/etc/apache2/sites-enabled# file puppetmaster puppetmaster: ASCII English text root@palladium:/etc/apache2/sites-enabled# if diff puppetmaster ../sites-available/puppetmaster ; then echo "meh, different content, confusing"; fi meh, different content, confusing Change-Id: I0e4aa880003370093c7ee1a49f71a16f697a83ef 14 August 2014, 00:24:50 UTC
563e43e svn - move Apache config from file to template Change-Id: Ia4b43d88578c6356393f60bd15c01d164fa98ac3 13 August 2014, 23:53:29 UTC
ec3aaaf ordered_yaml(): omit document header According to the YAML spec, 'If no explicit header line is specified at the start of the stream, the processor should behave as if a header line containing an unadorned “---” was specified.' (<http://yaml.org/spec/1.0/>). Since some applications barf on the document header, omit it. Change-Id: Id8163d372669b8d7fcae30e7cd40bbbbf419c7a7 13 August 2014, 23:17:39 UTC
c4fe671 wikistats - use apache::site use new apache::site method rather than file resources Change-Id: Iefbddfca39f0d9b41e3d9d3d50a0161769c33774 13 August 2014, 22:20:53 UTC
3164df7 ganglia - use apache::site use the newer apache::site method rather than a file resource to setup the Apache site Change-Id: I29f3c4e6d3454ce0ef89fe298f8449c92a33758b 13 August 2014, 22:05:31 UTC
b986a0b graphite - use apache::site use the newer apache::site method rather than a file resource to setup the Apache site Change-Id: I52fa1c76de155a45db6c48d3bb52f2bb85834167 13 August 2014, 21:59:47 UTC
652d569 tendril - use apache::site use the newer apache::site method rather than a file resource to setup the Apache site Change-Id: I24db3766742f4ff7517419ec7a8e42ccd23b1b28 13 August 2014, 21:48:47 UTC
a64123c openstack Apache conf, also listen on port 80 Change-Id: I5ad977f64c23cfda96d29e2c260173f16fe685ca 13 August 2014, 21:23:15 UTC
c7990e5 Merge "gdash - use apache::site" into production 13 August 2014, 20:53:50 UTC
3148671 Merge "openstack - use apache::conf for port" into production 13 August 2014, 20:50:14 UTC
9ed16b7 Merge "kibana - use apache::site" into production 13 August 2014, 20:46:32 UTC
1041eab openstack - use apache::conf for port use apache::conf to make it listen on 443, instead of a flat file at the same time this gets rid of the requirement that breaks in Ic484ea22541cd Change-Id: I4c11a2ea2b5f5bcfd3701459cdec23f78d28373e 13 August 2014, 20:43:06 UTC
263e6cf wmflib: add ordered_yaml() Ruby 1.9+ hashes maintain insertion order. The serialized form of hash is therefore stable across Puppet runs, *but* only insofar as the insertion order is stable. This patch does two things: first, it makes ordered_json.rb substantially simpler, by keys from a hash into a new hash in lexicographical order and then serializing the result. Second, it adds an ordered_yaml function that does the same thing, except it emits YAML rather than JSON. Change-Id: I62605c2a23b3607dd8616c25d823ee341fb327c7 13 August 2014, 20:35:53 UTC
8f5c32a wmflib: add validate_ensure() The alternative forms are all annoying to type and are used inconsistently: validate_re($ensure, '^(present|absent)$', "\$ensure must be..., etc.") if $ensure !~ /^(present|absent)$/ { fail("\$ensure must be..., etc." ) if !($ensure in ['present', 'absent']) ... So add this one, which will be faster than using regexes anyway: validate_ensure($ensure) Yes, different resource types take other values for 'ensure'; but 'present' / 'absent' are the obvious universals, so I don't think that this will cause confusion. Change-Id: I1b55c0d4f9dede5690a3e3e8b72f4428eb49de86 13 August 2014, 20:23:57 UTC
f3aca29 Merge "Revert "Revert "openstack - use apache::site""" into production 13 August 2014, 20:23:23 UTC
9147665 kibana - use apache::site use the newer apache::site method to setup the Apache site Change-Id: I32f3efbb421be2963fcbd79599b383fa13344a5d 13 August 2014, 20:22:36 UTC
5b1aad6 gdash - use apache::site use the newer apache::site method to setup the Apache site Change-Id: I54ae04770e847b434631ac00d7dec953516ec79e 13 August 2014, 20:18:45 UTC
d23199a mediawiki: create common-local directory and add a table-flip emoji, because that's how I feel when I reflect on the app server symlink clusterfuck. Rage. Change-Id: Ifc2f1646ee3716986f93965671a08f74315b994d 13 August 2014, 20:13:48 UTC
5a3b82e Revert "Revert "openstack - use apache::site"" This reverts commit 053369443ccb821a03aea5b0bdbe434b41a7457e. Change-Id: Ic484ea22541cd9199cdc53e8cd9916e3c3c46f80 13 August 2014, 19:18:37 UTC
f27758f Create icinga::monitor::nsca::client class This class installs nsca-client and /etc/send_nsca.cfg from the private repo Change-Id: I7c7400dd89e96a651933e3e3c3928bba22a5564b 13 August 2014, 19:09:31 UTC
0533694 Revert "openstack - use apache::site" This reverts commit 851859c7f5a932e1ad0eb75b7d6172639e1e493b. "Error: Failed to apply catalog: Could not find dependency File[/etc/apache2/sites-enabled/wikitech.wikimedia.org] for File[/etc/apache2/conf.d/ports-wikitech.conf] at /etc/puppet/manifests/openstack.pp:417" Change-Id: I00a9e2fea9129eca8ea8655084739523b300fe46 13 August 2014, 17:43:36 UTC
851859c openstack - use apache::site use the newer apache::site method rather than a file resource and have proper symlinks again currently, due to refactoring, we have: @virt1000:/# if diff -q /etc/apache2/sites-enabled/wikitech.wikimedia.org /etc/apache2/sites-available/wikitech.wikimedia.org; then echo "confusing to have 2 different files, where one is unused"; fi Files .. differ Change-Id: If77393a6e6453099ccc3b1fb41bd99d27b19500c 13 August 2014, 18:36:34 UTC
0e9aafa Set up passive icinga for webrequest data imports in HDFS and Hive Change-Id: I52ceb5e6a98240603bc8361f21124c8033e45bb7 13 August 2014, 17:17:19 UTC
b07a36d Remove redundant spam check bypass acl_m2 We already bypass it for all wikimedia networks Change-Id: Ia240dc65d295a0c173229c86640db7a042e96a16 13 August 2014, 17:11:52 UTC
5101798 apache: add a 'replaces' parameter to apache::conf This will allow us to remove, whenever we feel like we need to, to ensure another file (tipically a module config file) shipped by the distro will not be included in the apache config. Change-Id: I137a5d40a669ca651a039c98bcbf5ba1b08d286e Signed-off-by: Giuseppe Lavagetto <glavagetto@wikimedia.org> 13 August 2014, 15:45:58 UTC
86cb849 Tools: Include mediawiki::multimedia::fonts in exec_environ When I previously included mediawiki::multimedia::fonts in role::labs::tools::execnode to resolve bug #58740 in change I421685039b7ae9f73535ac92b3d8b6e5ecf1b1ac, I did so under the apparently wrong impression that only roles should include classes. The net effect however was that the fonts were only installed on "true" execution nodes and not on the bastion or webserver nodes, making for a rather strange setup. At user request and with more logic, this change moves the include to toollabs::exec_environ so that the fonts are installed on all nodes where they are actually expected and needed. Bug: 66354 Change-Id: Ie071002eef9d4824d7b70de0c6bdeb679450cca4 13 August 2014, 15:29:29 UTC
3d28009 Merge "Get rid of symlinks to scap scripts" into production 13 August 2014, 15:08:37 UTC
8ccd096 Get rid of symlinks to scap scripts Requires I40a891b1c Change-Id: If80560123f446163f0f08a431400984f303de82e 13 August 2014, 15:02:41 UTC
1b9cebf Add 'udplog' to $nova_dnsmasq_aliases Corresponding change in production: I33d3e4a58. Change-Id: I2a25eec4fa9744e1127efd46186f9d92d4cfd045 13 August 2014, 14:38:20 UTC
ca46a83 Stop forwarding udp2log's EventLogging data to universities Change-Id: I628545762d0c33a184571137c6b55bbd5636de24 13 August 2014, 13:30:37 UTC
fcf5123 puppetmaster: fix typo in scripts.pp Change-Id: Icdf8c04ba4daeedf9a612d4ee4397ec435f4b2d5 Signed-off-by: Giuseppe Lavagetto <glavagetto@wikimedia.org> 13 August 2014, 10:27:42 UTC
9115a16 puppetmaster: make reimaging servers easier. Change-Id: Id898cce1f634f4ae2015173f15e95ab94d60fc43 Signed-off-by: Giuseppe Lavagetto <glavagetto@wikimedia.org> 13 August 2014, 10:13:50 UTC
49ea85f osm planet import/sync, make sure we use hstore hstore is a key/value store in postgresql that is used by openstreetmap as a tagging scheme. Pass the -k parameter to osm2pgsql to use it Change-Id: I037057ea6e475dbac5f3ff424b72464f94e3f413 13 August 2014, 06:57:49 UTC
6751a5a tendril: add access for ldap groups ops and nda As the wmf is organisation specific and others also need access, the restriction should be replaced with the groups ops and nda. The group nda was newly created for cases like this. This still retains access for the wmf group because we don't know who from that group needs access. Change-Id: I1e8ccb91ee9985e0a6cf3076f31655bf1e823a0e 13 August 2014, 01:42:26 UTC
79b1c6e Remove mw1053 from mediawiki-installation dsh Scap etc. doesn't work there atm... Change-Id: Ica160668160dd2950b63378ccac454d6a2f1054c 12 August 2014, 23:28:17 UTC
2cad95b Fix kafka & udp2log filtering of ZERO The \d{3}-\d{2} syntax is incompatible with future plans for this field, and might break in some cases going forward. Expanding it to 'zero=\d' Change-Id: I121bb6c15767bbbade7f8cb10745ef75e3592bc2 12 August 2014, 20:10:37 UTC
f464c90 varnish/zero.inc.vcl.erb - retab Change-Id: If814c4307349ff412dd2268c6e8833702f9b24e4 12 August 2014, 19:50:13 UTC
caf701d New admin group for eventlogging troubleshooting. RT 7769 (nuria) & 8034 (qchris) And Nuria has had access to vanadium for the same things. This rolls it into one nice setup. Change-Id: Iecbea59a8f294167a8d2ca385609e1d4b2f0f46c 12 August 2014, 17:02:38 UTC
10cb6ca remove aluminium from sites.pp and dhcp config Change-Id: I9fe8a306ff9360acc89a98eca88826ddd784e5f7 12 August 2014, 16:39:22 UTC
0704a1e salt-minion service should fail to start if master rejects key salt-minion bails if the Salt master rejects the minion's key, but by this point the Upstart job has emitted the start/running signal, so Puppet thinks the service is up, and it executes things that require it, like various Trebuchet operations. These usually hang. So, perform a salt-call test.ping before starting salt-minion. Change-Id: I78da7fec86b7991f7c1119fa59dde0471a4471cb 12 August 2014, 15:56:38 UTC
76a2c38 Sanitarium sysv script + basic monitoring. Change-Id: I67c91975317ba02ec51912a6afbd8ffa3aa13f12 12 August 2014, 14:57:45 UTC
75bcddf MariaDB 10 in the Sanitarium (pre-labsdb) Change-Id: I26b03bbded727f5a5e4e901af808a0617577bfe5 12 August 2014, 13:51:36 UTC
b996e51 Merge "Small lint-fix for hhvm.pp" into production 12 August 2014, 12:40:35 UTC
eb75ea1 Make permanent some labsdb TokuDB settings applied with SET GLOBAL. Change-Id: Ib4e70b91cc871bdf5f3e6dd00a575451363181f6 12 August 2014, 11:50:42 UTC
a9eb347 Small lint-fix for hhvm.pp Change-Id: I3130327a5210d708b5ff41409424f9c9647e9900 12 August 2014, 11:42:10 UTC
18a3983 Enable mathoid deployment::target on beta This was needed anyway Change-Id: Idd36f111ee2fb47a0c602b6839cba716f5bd47b3 12 August 2014, 11:04:40 UTC
c839164 osm: Enable ganglia diskstat plugin Evaluate its usage in the osm databases Change-Id: I2e7a028047ea116068cf64341e049d1afee15a0a 12 August 2014, 08:13:42 UTC
6433773 Merge "enable gelf logging from hadoop workers" into production 11 August 2014, 21:41:50 UTC
37be54a Merge "rm old puppet_disabled check,replaced by new chk" into production 11 August 2014, 21:39:32 UTC
5e9fad6 rm old puppet_disabled check,replaced by new chk the new "puppet run" check should also detect those disabled puppet agents, so keeping the old one would just cause duplicate alarms and longer/slower Icinga config Change-Id: I46d8e23ea0a96395252dc9323cb24c49f70614cf 11 August 2014, 21:37:53 UTC
5529265 enable gelf logging from hadoop workers Change-Id: Ifd06469ccc5fbb5a603ef59f8c7458cf3e24049a 11 August 2014, 21:35:57 UTC
8df79cd Merge "Hadoop role: depend on JARs for GELF, pass param" into production 11 August 2014, 21:32:21 UTC
7d8c357 Hadoop role: depend on JARs for GELF, pass param * rework of https://gerrit.wikimedia.org/r/#/c/140677/ * logstash-gelf.jar is now packaged as liblogstash-gelf-java Change-Id: I129ee2c00d174c4c190321040afc52c65da38581 11 August 2014, 21:26:59 UTC
f73da31 Merge "remove deprecated config related to fundraising/aluminium" into production 11 August 2014, 21:19:00 UTC
3f6b6e3 remove deprecated config related to fundraising/aluminium Change-Id: Iade7f550838fa59ae27fefe0c9b1998798d26bd5 11 August 2014, 21:15:41 UTC
bc986f0 Zero: updated 436-01 436-04 - both unified, both support https Change-Id: I636d965f975cca539354d61d6c4494566f302bf3 11 August 2014, 21:13:00 UTC
back to top