From 9648940ed5d6bda1488794264bec59bbe58e7eb8 Mon Sep 17 00:00:00 2001 From: Lukas Bezdicka Date: Tue, 8 Dec 2015 18:13:56 +0100 Subject: [PATCH 01/35] Sync openstack modules with their stable/liberty release Update aodh to a09301ab15b621e5e485fb605ab488e64a059d2a a09301ab15b621e5e485fb605ab488e64a059d2a release: prepare 7.0.0 (liberty) fd27b60ce5d197e5394af03fec62a6f1784c2ead Merge "Remove class_parameter_defaults puppet-lint check" f601fedb7cca4a0b139e6702ee93a1751d0fc9c1 Acceptance: use common bits from puppet-openstack-integration bf7daddae745243b6b5cdf9b560481c168fb7f1f Remove class_parameter_defaults puppet-lint check cd0920ff960b4be9608a3b96555538feb260b925 Tag all Aodh packages 42c34763bb4266e5315618a59082dc580eb13039 Implement Listener service 6e30e9f4104d39d626d41c3e7fecbbc04dbac762 Implement Notifier service 78af0f02c3a8f21cfe28248705591c046972f0ce Implement Client class 9510ce46e4d9d59a44fdb4e60a072d6577992bf7 Implement Evaluator service ea89099ff7080056b6ec26668d981b70303eb302 Merge "auth: fix doc for default port" 2f4dca98d2e2c32aee314fc6d5f2ad119cb7f5f5 Merge "api: fix default port to bind API service" 1eeba2001b229dcecb6f034da9ba3ea9cb4360a4 db: Use postgresql lib class for psycopg package 3c50874c7a0d65d2e75589b6aa383ab0a41511fb auth: fix doc for default port fb8be800021cb24af00b69112e6e3acbe26eb7af api: fix default port to bind API service Change-Id: Ia59624de78b883f1da2ae43e06d32cafc8ed9fbf Update ceilometer to d303048dd29f8e645a992aa8ec23799bcc8eedfb d303048dd29f8e645a992aa8ec23799bcc8eedfb Add deprecation warning for Qpid rpc driver 8bc29454c17191f13101aaa59a513046ed732691 release: prepare 7.0.0 (liberty) 6add4485c28a039e04433a81739720e5af9c800e Fix unit tests against Puppet 4.3.0 Change-Id: I3a5a2abd44e0f7c38aae4e464d35cd1815479b30 Update cinder to 5add15bc80b8867b3d079f8aa563a609800f3e61 5add15bc80b8867b3d079f8aa563a609800f3e61 Merge "Add deprecation warning for Qpid rpc driver" into stable/liberty 82c5426f3bbe2609a8df187bfa2c23111f6add51 Add deprecation warning for Qpid rpc driver 701eea29fb5818c9e0ca94c05373bfed89e03a6a Add keymgr/encryption_auth_url parameter to cinder API 995c58730a0ff3dd63c8915ced5bddfcdfa084f1 Merge "release: prepare 7.0.0 (liberty)" 0d1d6b548852bd5d0ffe1fa3825f0772be23b0de Merge "Remove keystone directives from api-paste file" 312546c2b38c133b7b093853a3dd38dc0ac0a091 release: prepare 7.0.0 (liberty) cdc53ae7ec55190de9299dc2196d0ec07321a26c Remove keystone directives from api-paste file Change-Id: I9bed5e8aa6f86fae230869360e9a300058386649 Update glance to f4495d93a064ea8d1f5a023c156ab73e14e69198 f4495d93a064ea8d1f5a023c156ab73e14e69198 Add deprecation warning for Qpid rpc driver d4b7415014b3ca4700f267a8445faf8e68e1d61d Update Glance Swift backend configuration 0f384787e3603f5a02c9905ba3696aef31a69552 release: prepare 7.0.0 (liberty) Change-Id: Ie1814c81fdc2a0ef25bf66c313f4b41d5990c308 Update gnocchi to f658ca8213fa94ff9de5abee73720c271631bd4d f658ca8213fa94ff9de5abee73720c271631bd4d release: prepare 7.0.0 (liberty) a22687f4ed81af9de84bcbd5ff43591ad21dd349 Implement Gnocchi Statsd support Change-Id: I9fd8bc95661049ac7cb3710ff1c93fa85604bcad Update heat to fac1ae0e794c2f1ba66132e77f83f9e8f035dcdf fac1ae0e794c2f1ba66132e77f83f9e8f035dcdf Add deprecation warning for Qpid rpc driver 9567116722717b710e57a28d2a9e6c0f4c5e4d50 release: prepare 7.0.0 (liberty) Change-Id: I07c66ae8540ee3170e21cddd8776157761443702 Update horizon to 0d1398f418ee89eb5cf8e2718a447420d4915e2f 0d1398f418ee89eb5cf8e2718a447420d4915e2f Fix wrong use of os_package_type 29670f4f211b5b044cc654a3c46a348d194272c9 release: prepare 7.0.0 (liberty) Change-Id: Ibde81cace92a33ec91305f399d62bb8eb360fc99 Update ironic to 0e7546bd7810e10ed3d108a95e6d7673ad3a98ed 0e7546bd7810e10ed3d108a95e6d7673ad3a98ed Add deprecation warning for Qpid rpc driver a7d945f624f03ddbda17c523ff0bf6d63e919887 release: prepare 7.0.0 (liberty) cf565754a1237817b7728639e0bf30c9293772f7 Merge "Add BOOTIF=${mac} to the inspector iPXE template" 8d9730a593d0141dad80f511488c66583af8fc55 Add BOOTIF=${mac} to the inspector iPXE template Change-Id: I65d3de0d5ed31e4800d4434a65cc6939054ed532 Update manila to 068aa077b8698f900fec3a2e5a1690214b5acfd2 068aa077b8698f900fec3a2e5a1690214b5acfd2 Add deprecation warning for Qpid rpc driver b61124dd209405fc4b6459a62dbb1265cfac471a release: prepare 7.0.0 (liberty) cf8041398f5f99eba06cf6dedc61be887595517c enable sharev2 API capacity 3878c3d56e29a636d492f500a82bcbc21b274666 Acceptance: use common bits from puppet-openstack-integration Change-Id: Ife246ae06959c1215762d0c0917648562e5bd72b Update neutron to 0f2cb57696705d3f9dc6f7b4c94f9b469b673486 0f2cb57696705d3f9dc6f7b4c94f9b469b673486 Merge "Add deprecation warning for Qpid rpc driver" into stable/liberty be8d51171d61384c99c34b673c8664c1cc36cc58 Add deprecation warning for Qpid rpc driver e81bc875fd2053ba6b82f8d1e87e115d81f957d9 Add parameters for Neutron QoS support 27d9bb556a9caffd100ada119ec3b0a96b3974ce release: prepare 7.0.0 (liberty) 0efb751408dac392153bd4a48a76cebc4098b6bc Fix unit tests against Puppet 4.3.0 Change-Id: Ie146d927d99afb5a2cbf9f21e00c162f8874caf0 Update nova to a495b4010d69adcc36927c95036b7d854a785700 a495b4010d69adcc36927c95036b7d854a785700 Add deprecation warning for Qpid rpc driver f7ae97b6285e27511c16eecc206c7301f1ba03f2 release: prepare 7.0.0 (liberty) Change-Id: Ibb66c78ec74596c7e1ec782131022be491daabac Update openstack_extras to b689f85d4f8d81eb8d1d1fc26a9f48acf71c1c41 b689f85d4f8d81eb8d1d1fc26a9f48acf71c1c41 release: prepare 7.0.0 (liberty) Change-Id: I0f60ec2b3dda0a6e3081d4a0118263b8bf830b62 Update swift to 5ca8b0b75f89e31be24d6ec45dfcb2bb38a93bdb 5ca8b0b75f89e31be24d6ec45dfcb2bb38a93bdb release: prepare 7.0.0 (liberty) Change-Id: If50f6dd86867bcb19d4adae5a66fe5a4e60ef1e5 Update tempest to a26f81393bd829ddbc8ca8e1e8de226538208f1e a26f81393bd829ddbc8ca8e1e8de226538208f1e release: prepare 7.0.0 (liberty) Change-Id: I0a4d3947e942070b8675e47344642a07b7e116cf Update trove to 02295f18915107203ef79ef37fe5cc284506cfc1 02295f18915107203ef79ef37fe5cc284506cfc1 Add deprecation warning for Qpid rpc driver 2462480f91102088928f18178f8db7f0d15429fa release: prepare 7.0.0 (liberty) Change-Id: I9a0fec54db7cd67511c26598c6f23b57d75fe634 Update vswitch to c374840910c823f7669cf2e1229c7df7192ae880 c374840910c823f7669cf2e1229c7df7192ae880 release: prepare 3.0.0 (liberty) Change-Id: Id54d59233109ba1b95e76c24fca33408137a8553 Update tripleo to 74b589497c9efe400403f2d446aa6335abdb89d5 74b589497c9efe400403f2d446aa6335abdb89d5 Merge "Fix unit tests failing against Puppet 4.3.x" f2d544fdb345cc4a303f45c4785752a98b4d54a6 Fix unit tests failing against Puppet 4.3.x 6021323d41c19e4aac4a3f353c1de9278763b548 Remove `validate_ip_address` validator. 4666fbad534f04b6dcb87923b061a4b1da6c1033 Merge "Fix unit tests failing against Puppet 4.3.0" 43574dadf192a1df942898a626a359f60f286a20 Merge "MidoNet services manifests" dafdc1f4efde8c57a8c7160ae0162866496089dc Merge "Set tunnel timeout for nova_novncproxy" 43dc589165250a3948df4d31ef5817fa9efbc19e Fix unit tests failing against Puppet 4.3.0 95c87ea79bee85373c68befb43f46d975dafe241 MidoNet services manifests 4945e2f4d321b31a0dec3ce0a53cf88fcbc41591 Set tunnel timeout for nova_novncproxy Change-Id: I2a1555ff352c896ee76724c7731011e7b22fed1e --- Puppetfile | 38 +++--- aodh/CHANGELOG.md | 4 + aodh/README.md | 2 + aodh/Rakefile | 4 +- aodh/examples/aodh.pp | 7 ++ aodh/manifests/api.pp | 4 +- aodh/manifests/auth.pp | 73 +++++++++++ aodh/manifests/client.pp | 22 ++++ aodh/manifests/db.pp | 5 +- aodh/manifests/db/mysql.pp | 2 +- aodh/manifests/db/postgresql.pp | 2 +- aodh/manifests/db/sync.pp | 23 ++-- aodh/manifests/evaluator.pp | 60 +++++++++ aodh/manifests/keystone/auth.pp | 6 +- aodh/manifests/listener.pp | 49 ++++++++ aodh/manifests/notifier.pp | 49 ++++++++ aodh/manifests/params.pp | 2 + aodh/metadata.json | 4 +- aodh/spec/acceptance/aodh_wsgi_apache_spec.rb | 94 ++++---------- aodh/spec/classes/aodh_api_spec.rb | 2 +- aodh/spec/classes/aodh_auth_spec.rb | 55 ++++++++ aodh/spec/classes/aodh_client_spec.rb | 33 +++++ aodh/spec/classes/aodh_db_spec.rb | 96 ++++++++++++++ aodh/spec/classes/aodh_evaluator_spec.rb | 113 +++++++++++++++++ aodh/spec/classes/aodh_listener_spec.rb | 99 +++++++++++++++ aodh/spec/classes/aodh_notifier_spec.rb | 99 +++++++++++++++ aodh/spec/spec_helper_acceptance.rb | 2 +- ceilometer/CHANGELOG.md | 47 +++++++ ceilometer/README.md | 2 +- ceilometer/Rakefile | 2 +- ceilometer/manifests/init.pp | 2 + ceilometer/metadata.json | 6 +- .../spec/classes/ceilometer_api_spec.rb | 13 +- ceilometer/spec/spec_helper_acceptance.rb | 2 +- cinder/CHANGELOG.md | 52 ++++++++ cinder/README.md | 2 +- cinder/Rakefile | 2 +- cinder/manifests/api.pp | 89 ++++++------- cinder/manifests/init.pp | 2 + cinder/manifests/qpid.pp | 2 + cinder/metadata.json | 6 +- cinder/spec/classes/cinder_api_spec.rb | 70 ++++++----- cinder/spec/spec_helper_acceptance.rb | 2 +- glance/CHANGELOG.md | 41 ++++++ glance/README.md | 2 +- glance/Rakefile | 2 +- glance/manifests/backend/swift.pp | 35 +++--- glance/manifests/notify/qpid.pp | 2 + glance/metadata.json | 6 +- .../spec/classes/glance_backend_swift_spec.rb | 42 +++---- glance/spec/spec_helper_acceptance.rb | 2 +- gnocchi/CHANGELOG.md | 4 + gnocchi/README.md | 2 + gnocchi/Rakefile | 2 +- gnocchi/examples/site.pp | 8 ++ gnocchi/manifests/api.pp | 2 +- gnocchi/manifests/params.pp | 4 + gnocchi/manifests/statsd.pp | 82 ++++++++++++ gnocchi/metadata.json | 6 +- gnocchi/spec/acceptance/basic_gnocchi_spec.rb | 7 ++ gnocchi/spec/classes/gnocchi_statsd_spec.rb | 103 +++++++++++++++ gnocchi/spec/spec_helper_acceptance.rb | 2 +- heat/CHANGELOG.md | 43 +++++++ heat/README.md | 2 +- heat/Rakefile | 2 +- heat/manifests/init.pp | 2 + heat/metadata.json | 6 +- heat/spec/spec_helper_acceptance.rb | 2 +- horizon/CHANGELOG.md | 31 +++++ horizon/README.md | 2 +- horizon/Rakefile | 2 +- horizon/manifests/init.pp | 2 +- horizon/metadata.json | 2 +- horizon/spec/classes/horizon_init_spec.rb | 6 +- horizon/spec/spec_helper_acceptance.rb | 2 +- ironic/CHANGELOG.md | 50 ++++++++ ironic/README.md | 2 +- ironic/Rakefile | 2 +- ironic/manifests/init.pp | 3 + ironic/metadata.json | 6 +- ironic/spec/spec_helper_acceptance.rb | 2 +- ironic/templates/inspector_ipxe.erb | 2 +- manila/CHANGELOG.md | 34 +++++ manila/README.md | 2 +- manila/Rakefile | 2 +- manila/manifests/init.pp | 2 + manila/manifests/keystone/auth.pp | 98 ++++++++++++--- manila/manifests/qpid.pp | 2 + manila/metadata.json | 8 +- manila/spec/acceptance/basic_manila_spec.rb | 81 ++---------- .../spec/classes/manila_keystone_auth_spec.rb | 45 ++++++- manila/spec/spec_helper_acceptance.rb | 2 +- neutron/CHANGELOG.md | 71 +++++++++++ neutron/README.md | 2 +- neutron/Rakefile | 2 +- neutron/manifests/agents/ml2/ovs.pp | 6 + neutron/manifests/agents/ml2/sriov.pp | 7 +- neutron/manifests/init.pp | 5 +- neutron/manifests/plugins/ml2.pp | 7 ++ neutron/manifests/server.pp | 7 ++ neutron/metadata.json | 10 +- .../spec/classes/neutron_agents_dhcp_spec.rb | 5 +- .../spec/classes/neutron_agents_l3_spec.rb | 5 +- .../spec/classes/neutron_agents_lbaas_spec.rb | 5 +- .../classes/neutron_agents_metadata_spec.rb | 7 +- .../classes/neutron_agents_metering_spec.rb | 5 +- .../neutron_agents_ml2_linuxbridge_spec.rb | 5 +- .../classes/neutron_agents_ml2_ovs_spec.rb | 16 ++- .../classes/neutron_agents_ml2_sriov_spec.rb | 16 ++- neutron/spec/classes/neutron_client_spec.rb | 5 +- neutron/spec/classes/neutron_db_spec.rb | 2 +- neutron/spec/classes/neutron_init_spec.rb | 9 +- .../classes/neutron_plugins_cisco_spec.rb | 5 +- .../classes/neutron_plugins_midonet_spec.rb | 5 +- ...on_plugins_ml2_bigswitch_restproxy_spec.rb | 5 +- .../neutron_plugins_ml2_bigswitch_spec.rb | 5 +- .../neutron_plugins_ml2_cisco_nexus_spec.rb | 5 +- .../classes/neutron_plugins_ml2_cisco_spec.rb | 5 +- ...plugins_ml2_cisco_type_nexus_vxlan_spec.rb | 5 +- .../neutron_plugins_ml2_cisco_ucsm_spec.rb | 5 +- .../spec/classes/neutron_plugins_ml2_spec.rb | 16 ++- .../classes/neutron_plugins_nuage_spec.rb | 10 +- .../spec/classes/neutron_plugins_nvp_spec.rb | 5 +- .../neutron_plugins_opencontrail_spec.rb | 5 +- .../classes/neutron_plugins_plumgrid_spec.rb | 5 +- neutron/spec/classes/neutron_policy_spec.rb | 5 +- .../neutron_server_notifications_spec.rb | 5 +- neutron/spec/classes/neutron_server_spec.rb | 38 ++++-- .../classes/neutron_services_fwaas_spec.rb | 5 +- neutron/spec/spec_helper_acceptance.rb | 2 +- nova/CHANGELOG.md | 60 +++++++++ nova/README.md | 2 +- nova/Rakefile | 2 +- nova/manifests/init.pp | 3 + nova/manifests/qpid.pp | 2 + nova/metadata.json | 10 +- nova/spec/spec_helper_acceptance.rb | 2 +- openstack_extras/CHANGELOG.md | 19 +++ openstack_extras/README.md | 2 +- openstack_extras/Rakefile | 2 +- openstack_extras/metadata.json | 2 +- .../spec/spec_helper_acceptance.rb | 2 +- swift/CHANGELOG.md | 40 ++++++ swift/README.md | 2 +- swift/Rakefile | 2 +- swift/metadata.json | 4 +- swift/spec/spec_helper_acceptance.rb | 2 +- tempest/CHANGELOG.md | 24 ++++ tempest/README.markdown | 2 +- tempest/Rakefile | 2 +- tempest/metadata.json | 2 +- tempest/spec/spec_helper_acceptance.rb | 2 +- tripleo/.fixtures.yml | 21 ++++ .../lib/puppet/parser/functions/extract_id.rb | 14 +++ .../functions/list_to_zookeeper_hash.rb | 24 ++++ tripleo/manifests/cluster/cassandra.pp | 69 +++++++++++ tripleo/manifests/cluster/zookeeper.pp | 65 ++++++++++ tripleo/manifests/loadbalancer.pp | 1 + tripleo/manifests/network/midonet/agent.pp | 61 +++++++++ tripleo/manifests/network/midonet/api.pp | 117 ++++++++++++++++++ .../classes/tripleo_cluster_cassandra_spec.rb | 45 +++++++ .../classes/tripleo_cluster_zookeeper_spec.rb | 115 +++++++++++++++++ tripleo/spec/classes/tripleo_firewall_spec.rb | 5 +- .../classes/tripleo_midonet_agent_spec.rb | 62 ++++++++++ .../spec/classes/tripleo_midonet_api_spec.rb | 72 +++++++++++ tripleo/spec/spec_helper.rb | 4 + trove/CHANGELOG.md | 30 +++++ trove/README.md | 2 +- trove/Rakefile | 2 +- trove/manifests/api.pp | 3 + trove/manifests/conductor.pp | 3 + trove/manifests/guestagent.pp | 3 + trove/manifests/taskmanager.pp | 3 + trove/metadata.json | 6 +- trove/spec/spec_helper_acceptance.rb | 2 +- vswitch/CHANGELOG.md | 20 +++ vswitch/README.md | 2 +- vswitch/metadata.json | 6 +- vswitch/spec/spec_helper_acceptance.rb | 2 +- 179 files changed, 2855 insertions(+), 478 deletions(-) create mode 100644 aodh/CHANGELOG.md create mode 100644 aodh/manifests/auth.pp create mode 100644 aodh/manifests/client.pp create mode 100644 aodh/manifests/evaluator.pp create mode 100644 aodh/manifests/listener.pp create mode 100644 aodh/manifests/notifier.pp create mode 100644 aodh/spec/classes/aodh_auth_spec.rb create mode 100644 aodh/spec/classes/aodh_client_spec.rb create mode 100644 aodh/spec/classes/aodh_db_spec.rb create mode 100644 aodh/spec/classes/aodh_evaluator_spec.rb create mode 100644 aodh/spec/classes/aodh_listener_spec.rb create mode 100644 aodh/spec/classes/aodh_notifier_spec.rb create mode 100644 gnocchi/CHANGELOG.md create mode 100644 gnocchi/manifests/statsd.pp create mode 100644 gnocchi/spec/classes/gnocchi_statsd_spec.rb create mode 100644 tripleo/lib/puppet/parser/functions/extract_id.rb create mode 100644 tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb create mode 100644 tripleo/manifests/cluster/cassandra.pp create mode 100644 tripleo/manifests/cluster/zookeeper.pp create mode 100644 tripleo/manifests/network/midonet/agent.pp create mode 100644 tripleo/manifests/network/midonet/api.pp create mode 100644 tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb create mode 100644 tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb create mode 100644 tripleo/spec/classes/tripleo_midonet_agent_spec.rb create mode 100644 tripleo/spec/classes/tripleo_midonet_api_spec.rb diff --git a/Puppetfile b/Puppetfile index 612b33131..e22f40bef 100644 --- a/Puppetfile +++ b/Puppetfile @@ -1,5 +1,5 @@ mod 'aodh', - :commit => '54f857d36bfa8567e9f6f5f4f8d778377229c702', + :commit => 'a09301ab15b621e5e485fb605ab488e64a059d2a', :git => 'https://github.com/openstack/puppet-aodh.git' mod 'apache', @@ -15,7 +15,7 @@ mod 'cassandra', :git => 'https://github.com/locp/cassandra.git' mod 'ceilometer', - :commit => '9869fbee668d35ff1f1838bec6545b1e13c0bfe3', + :commit => 'd303048dd29f8e645a992aa8ec23799bcc8eedfb', :git => 'https://github.com/openstack/puppet-ceilometer.git' mod 'ceph', @@ -27,7 +27,7 @@ mod 'certmonger', :git => 'https://github.com/rcritten/puppet-certmonger.git' mod 'cinder', - :commit => '181de2ef5119afab5157c290f10e9750507d95d2', + :commit => '5add15bc80b8867b3d079f8aa563a609800f3e61', :git => 'https://github.com/openstack/puppet-cinder.git' mod 'common', @@ -63,7 +63,7 @@ mod 'galera', :git => 'https://github.com/redhat-openstack/puppet-galera.git' mod 'glance', - :commit => 'da26542fbbdcf93e454f3c91313c7c2f778f63d0', + :commit => 'f4495d93a064ea8d1f5a023c156ab73e14e69198', :git => 'https://github.com/openstack/puppet-glance.git' mod 'gluster', @@ -71,7 +71,7 @@ mod 'gluster', :git => 'https://github.com/purpleidea/puppet-gluster.git' mod 'gnocchi', - :commit => '3594acae464c01f2475daf853b10ce777e5c3ed0', + :commit => 'f658ca8213fa94ff9de5abee73720c271631bd4d', :git => 'https://github.com/openstack/puppet-gnocchi.git' mod 'haproxy', @@ -79,11 +79,11 @@ mod 'haproxy', :git => 'https://github.com/puppetlabs/puppetlabs-haproxy.git' mod 'heat', - :commit => '35bf220a375e42ecb175481aa4d97839b4a4e382', + :commit => 'fac1ae0e794c2f1ba66132e77f83f9e8f035dcdf', :git => 'https://github.com/openstack/puppet-heat.git' mod 'horizon', - :commit => '93b54f518f4eb283191478d5b2d0b1f9fd9e6389', + :commit => '0d1398f418ee89eb5cf8e2718a447420d4915e2f', :git => 'https://github.com/openstack/puppet-horizon.git' mod 'inifile', @@ -95,7 +95,7 @@ mod 'ipa', :git => 'https://github.com/xbezdick/puppet-ipa-1.git' mod 'ironic', - :commit => '5f9c8fc38485e9d974a889de7ea7cbbb1c51fa3c', + :commit => '0e7546bd7810e10ed3d108a95e6d7673ad3a98ed', :git => 'https://github.com/openstack/puppet-ironic.git' mod 'java', @@ -119,7 +119,7 @@ mod 'kibana3', :git => 'https://github.com/thejandroman/puppet-kibana3.git' mod 'manila', - :commit => 'b77f1736bc0221acb433b5ffb6aa8f291d83ff13', + :commit => '068aa077b8698f900fec3a2e5a1690214b5acfd2', :git => 'https://github.com/openstack/puppet-manila.git' mod 'memcached', @@ -155,11 +155,11 @@ mod 'nagios', :git => 'https://github.com/gildub/puppet-nagios-openstack.git' mod 'neutron', - :commit => '11f05fff53f7ec9964494d63def3db37fdf9da39', + :commit => '0f2cb57696705d3f9dc6f7b4c94f9b469b673486', :git => 'https://github.com/openstack/puppet-neutron.git' mod 'nova', - :commit => '5d42ed567197928e09d731c545877a6b957e04b1', + :commit => 'a495b4010d69adcc36927c95036b7d854a785700', :git => 'https://github.com/openstack/puppet-nova.git' mod 'nssdb', @@ -175,7 +175,7 @@ mod 'opendaylight', :git => 'https://github.com/dfarrell07/puppet-opendaylight.git' mod 'openstack_extras', - :commit => 'cdeede97b90b5bbb0a54125ec68c36cf7e249370', + :commit => 'b689f85d4f8d81eb8d1d1fc26a9f48acf71c1c41', :git => 'https://github.com/openstack/puppet-openstack_extras.git' mod 'openstacklib', @@ -211,10 +211,6 @@ mod 'rsync', :git => 'https://github.com/puppetlabs/puppetlabs-rsync.git' mod 'sahara', - :commit => '6862557a4db48b01176e30be6877fda5503f085b', - :git => 'https://github.com/openstack/puppet-sahara.git' - -mod 'sensu' :commit => 'bc5e501e9b6c7ac38181c8749fb6a04b24efdb94', :git => 'https://github.com/sensu/sensu-puppet.git' @@ -235,7 +231,7 @@ mod 'stdlib', :git => 'https://github.com/puppetlabs/puppetlabs-stdlib.git' mod 'swift', - :commit => '6145bf61206381643cdcaaabc285f93d75c0c2e4', + :commit => '5ca8b0b75f89e31be24d6ec45dfcb2bb38a93bdb', :git => 'https://github.com/openstack/puppet-swift.git' mod 'sysctl', @@ -243,7 +239,7 @@ mod 'sysctl', :git => 'https://github.com/puppetlabs/puppetlabs-sysctl.git' mod 'tempest', - :commit => 'e6edd0871739c0dfe6cdc2b22fa5de3ba269f401', + :commit => 'a26f81393bd829ddbc8ca8e1e8de226538208f1e', :git => 'https://github.com/openstack/puppet-tempest.git' mod 'timezone', @@ -255,11 +251,11 @@ mod 'tomcat', :git => 'https://github.com/puppetlabs/puppetlabs-tomcat.git' mod 'tripleo', - :commit => 'ae595ce7731f2b286d4ffd280ed735b61d86b63c', + :commit => '74b589497c9efe400403f2d446aa6335abdb89d5', :git => 'https://github.com/openstack/puppet-tripleo.git' mod 'trove', - :commit => '6d541be368a09f73a83ef598443395a4bf811f82', + :commit => '02295f18915107203ef79ef37fe5cc284506cfc1', :git => 'https://github.com/openstack/puppet-trove' mod 'tuskar', @@ -279,7 +275,7 @@ mod 'vlan', :git => 'https://github.com/derekhiggins/puppet-vlan.git' mod 'vswitch', - :commit => 'ae87d9b0f8f7dbba9151312fb5205e89caf61109', + :commit => 'c374840910c823f7669cf2e1229c7df7192ae880', :git => 'https://github.com/openstack/puppet-vswitch.git' mod 'xinetd', diff --git a/aodh/CHANGELOG.md b/aodh/CHANGELOG.md new file mode 100644 index 000000000..1185c957a --- /dev/null +++ b/aodh/CHANGELOG.md @@ -0,0 +1,4 @@ +##2015-11-24 - 7.0.0 +###Summary + +- Initial release of the puppet-aodh module diff --git a/aodh/README.md b/aodh/README.md index 234a0a87d..969aa33d3 100644 --- a/aodh/README.md +++ b/aodh/README.md @@ -1,6 +1,8 @@ AODH ==== +7.0.0 - 2015.2.0 - Liberty + #### Table of Contents 1. [Overview - What is the AODH module?](#overview) diff --git a/aodh/Rakefile b/aodh/Rakefile index 3c3603e3c..9692ffdd2 100644 --- a/aodh/Rakefile +++ b/aodh/Rakefile @@ -15,7 +15,7 @@ PuppetLint::RakeTask.new :lint do |config| config.ignore_paths = ["spec/**/*.pp", "vendor/**/*.pp"] config.fail_on_warnings = true config.log_format = '%{path}:%{linenumber}:%{KIND}: %{message}' - config.disable_checks = ["80chars", "class_inherits_from_params_class", "class_parameter_defaults", "only_variable_string"] + config.disable_checks = ["80chars", "class_inherits_from_params_class", "only_variable_string"] end desc "Run acceptance tests" @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/aodh/examples/aodh.pp b/aodh/examples/aodh.pp index 26cbdf1c0..1dcb65eb8 100644 --- a/aodh/examples/aodh.pp +++ b/aodh/examples/aodh.pp @@ -9,3 +9,10 @@ class { '::aodh::wsgi::apache': ssl => false, } +class { '::aodh::auth': + auth_password => 'a_big_secret', +} +class { '::aodh::evaluator': } +class { '::aodh::notifier': } +class { '::aodh::listener': } +class { '::aodh::client': } diff --git a/aodh/manifests/api.pp b/aodh/manifests/api.pp index 3b9576569..7d7f5044b 100644 --- a/aodh/manifests/api.pp +++ b/aodh/manifests/api.pp @@ -36,7 +36,7 @@ # # [*port*] # (optional) The aodh api port. -# Defaults to 8777 +# Defaults to 8042 # # [*package_ensure*] # (optional) ensure state for package. @@ -61,7 +61,7 @@ $keystone_auth_uri = false, $keystone_identity_uri = false, $host = '0.0.0.0', - $port = '8777', + $port = '8042', $service_name = $::aodh::params::api_service_name, ) inherits aodh::params { diff --git a/aodh/manifests/auth.pp b/aodh/manifests/auth.pp new file mode 100644 index 000000000..d83551f16 --- /dev/null +++ b/aodh/manifests/auth.pp @@ -0,0 +1,73 @@ +# The aodh::auth class helps configure auth settings +# +# == Parameters +# [*auth_url*] +# the keystone public endpoint +# Optional. Defaults to 'http://localhost:5000/v2.0' +# +# [*auth_region*] +# the keystone region of this node +# Optional. Defaults to 'RegionOne' +# +# [*auth_user*] +# the keystone user for aodh services +# Optional. Defaults to 'aodh' +# +# [*auth_password*] +# the keystone password for aodh services +# Required. +# +# [*auth_tenant_name*] +# the keystone tenant name for aodh services +# Optional. Defaults to 'services' +# +# [*auth_tenant_id*] +# the keystone tenant id for aodh services. +# Optional. Defaults to undef. +# +# [*auth_cacert*] +# Certificate chain for SSL validation. Optional; Defaults to 'undef' +# +# [*auth_endpoint_type*] +# Type of endpoint in Identity service catalog to use for +# communication with OpenStack services. +# Optional. Defaults to undef. +# +class aodh::auth ( + $auth_password, + $auth_url = 'http://localhost:5000/v2.0', + $auth_region = 'RegionOne', + $auth_user = 'aodh', + $auth_tenant_name = 'services', + $auth_tenant_id = undef, + $auth_cacert = undef, + $auth_endpoint_type = undef, +) { + + if $auth_cacert { + aodh_config { 'service_credentials/os_cacert': value => $auth_cacert } + } else { + aodh_config { 'service_credentials/os_cacert': ensure => absent } + } + + aodh_config { + 'service_credentials/os_auth_url' : value => $auth_url; + 'service_credentials/os_region_name' : value => $auth_region; + 'service_credentials/os_username' : value => $auth_user; + 'service_credentials/os_password' : value => $auth_password, secret => true; + 'service_credentials/os_tenant_name' : value => $auth_tenant_name; + } + + if $auth_tenant_id { + aodh_config { + 'service_credentials/os_tenant_id' : value => $auth_tenant_id; + } + } + + if $auth_endpoint_type { + aodh_config { + 'service_credentials/os_endpoint_type' : value => $auth_endpoint_type; + } + } + +} diff --git a/aodh/manifests/client.pp b/aodh/manifests/client.pp new file mode 100644 index 000000000..ebffe6c76 --- /dev/null +++ b/aodh/manifests/client.pp @@ -0,0 +1,22 @@ +# +# Installs the aodh python library. +# +# == parameters +# [*ensure*] +# ensure state for pachage. +# +class aodh::client ( + $ensure = 'present' +) { + + include ::aodh::params + + # there is no aodhclient yet + package { 'python-ceilometerclient': + ensure => $ensure, + name => $::aodh::params::client_package_name, + tag => 'openstack', + } + +} + diff --git a/aodh/manifests/db.pp b/aodh/manifests/db.pp index 6d5d521b3..350c84a8e 100644 --- a/aodh/manifests/db.pp +++ b/aodh/manifests/db.pp @@ -43,6 +43,8 @@ $database_max_overflow = 20, ) { + include ::aodh::params + $database_connection_real = pick($::aodh::database_connection, $database_connection) $database_idle_timeout_real = pick($::aodh::database_idle_timeout, $database_idle_timeout) $database_min_pool_size_real = pick($::aodh::database_min_pool_size, $database_min_pool_size) @@ -62,7 +64,8 @@ require 'mysql::bindings::python' } /^postgresql:\/\//: { - $backend_package = $::aodh::params::psycopg_package_name + $backend_package = false + require 'postgresql::lib::python' } /^sqlite:\/\//: { $backend_package = $::aodh::params::sqlite_package_name diff --git a/aodh/manifests/db/mysql.pp b/aodh/manifests/db/mysql.pp index 71078e1bf..6cda0479f 100644 --- a/aodh/manifests/db/mysql.pp +++ b/aodh/manifests/db/mysql.pp @@ -65,5 +65,5 @@ allowed_hosts => $allowed_hosts, } - ::Openstacklib::Db::Mysql['aodh'] ~> Exec<| title == 'aodh-manage db_sync' |> + ::Openstacklib::Db::Mysql['aodh'] ~> Exec<| title == 'aodh-db-sync' |> } diff --git a/aodh/manifests/db/postgresql.pp b/aodh/manifests/db/postgresql.pp index 86e645b41..d34aa9d93 100644 --- a/aodh/manifests/db/postgresql.pp +++ b/aodh/manifests/db/postgresql.pp @@ -50,6 +50,6 @@ privileges => $privileges, } - ::Openstacklib::Db::Postgresql['aodh'] ~> Exec<| title == 'aodh-manage db_sync' |> + ::Openstacklib::Db::Postgresql['aodh'] ~> Exec<| title == 'aodh-db-sync' |> } diff --git a/aodh/manifests/db/sync.pp b/aodh/manifests/db/sync.pp index 27aa0f80c..a6566c741 100644 --- a/aodh/manifests/db/sync.pp +++ b/aodh/manifests/db/sync.pp @@ -1,14 +1,23 @@ # -# Class to execute "aodh-manage db_sync +# Class to execute "aodh-dbsync" # -class aodh::db::sync { - exec { 'aodh-manage db_sync': +# [*user*] +# (optional) User to run dbsync command. +# Defaults to 'aodh' +# +class aodh::db::sync ( + $user = 'aodh', +){ + exec { 'aodh-db-sync': + command => 'aodh-dbsync --config-file /etc/aodh/aodh.conf', path => '/usr/bin', - user => 'aodh', refreshonly => true, - subscribe => [Package['aodh'], Aodh_config['database/connection']], - require => User['aodh'], + user => $user, + logoutput => on_failure, } - Exec['aodh-manage db_sync'] ~> Service<| title == 'aodh' |> + Package<| tag == 'aodh-package' |> ~> Exec['aodh-db-sync'] + Exec['aodh-db-sync'] ~> Service<| tag == 'aodh-db-sync-service' |> + Aodh_config<||> ~> Exec['aodh-db-sync'] + Aodh_config<| title == 'database/connection' |> ~> Exec['aodh-db-sync'] } diff --git a/aodh/manifests/evaluator.pp b/aodh/manifests/evaluator.pp new file mode 100644 index 000000000..7270b79b5 --- /dev/null +++ b/aodh/manifests/evaluator.pp @@ -0,0 +1,60 @@ +# Installs the aodh evaluator service +# +# == Params +# [*enabled*] +# (optional) Should the service be enabled. +# Defaults to true. +# +# [*manage_service*] +# (optional) Whether the service should be managed by Puppet. +# Defaults to true. +# +# [*package_ensure*] +# (optional) ensure state for package. +# Defaults to 'present' +# +# [*coordination_url*] +# (optional) The url to use for distributed group membership coordination. +# Defaults to undef. +# +class aodh::evaluator ( + $manage_service = true, + $enabled = true, + $package_ensure = 'present', + $coordination_url = undef, +) { + + include ::aodh::params + + Aodh_config<||> ~> Service['aodh-evaluator'] + + if $coordination_url { + aodh_config { + 'coordination/backend_url' : value => $coordination_url; + } + } + + Package[$::aodh::params::evaluator_package_name] -> Service['aodh-evaluator'] + ensure_resource( 'package', [$::aodh::params::evaluator_package_name], + { ensure => $package_ensure, + tag => ['openstack', 'aodh-package'] } + ) + + if $manage_service { + if $enabled { + $service_ensure = 'running' + } else { + $service_ensure = 'stopped' + } + } + + Package['aodh'] -> Service['aodh-evaluator'] + service { 'aodh-evaluator': + ensure => $service_ensure, + name => $::aodh::params::evaluator_service_name, + enable => $enabled, + hasstatus => true, + hasrestart => true, + tag => ['aodh-service','aodh-db-sync-service'] + } +} diff --git a/aodh/manifests/keystone/auth.pp b/aodh/manifests/keystone/auth.pp index 9a2747987..d5f83fffc 100644 --- a/aodh/manifests/keystone/auth.pp +++ b/aodh/manifests/keystone/auth.pp @@ -38,15 +38,15 @@ # Defaults to the value of auth_name. # # [*public_url*] -# (optional) The endpoint's public url. (Defaults to 'http://127.0.0.1:9311') +# (optional) The endpoint's public url. (Defaults to 'http://127.0.0.1:8042') # This url should *not* contain any trailing '/'. # # [*admin_url*] -# (optional) The endpoint's admin url. (Defaults to 'http://127.0.0.1:9311') +# (optional) The endpoint's admin url. (Defaults to 'http://127.0.0.1:8042') # This url should *not* contain any trailing '/'. # # [*internal_url*] -# (optional) The endpoint's internal url. (Defaults to 'http://127.0.0.1:9311') +# (optional) The endpoint's internal url. (Defaults to 'http://127.0.0.1:8042') # This url should *not* contain any trailing '/'. # class aodh::keystone::auth ( diff --git a/aodh/manifests/listener.pp b/aodh/manifests/listener.pp new file mode 100644 index 000000000..7de3b4b65 --- /dev/null +++ b/aodh/manifests/listener.pp @@ -0,0 +1,49 @@ +# Installs the aodh listener service +# +# == Params +# [*enabled*] +# (optional) Should the service be enabled. +# Defaults to true. +# +# [*manage_service*] +# (optional) Whether the service should be managed by Puppet. +# Defaults to true. +# +# [*package_ensure*] +# (optional) ensure state for package. +# Defaults to 'present' +# +class aodh::listener ( + $manage_service = true, + $enabled = true, + $package_ensure = 'present', +) { + + include ::aodh::params + + Aodh_config<||> ~> Service['aodh-listener'] + + Package[$::aodh::params::listener_package_name] -> Service['aodh-listener'] + ensure_resource( 'package', [$::aodh::params::listener_package_name], + { ensure => $package_ensure, + tag => ['openstack', 'aodh-package'] } + ) + + if $manage_service { + if $enabled { + $service_ensure = 'running' + } else { + $service_ensure = 'stopped' + } + } + + Package['aodh'] -> Service['aodh-listener'] + service { 'aodh-listener': + ensure => $service_ensure, + name => $::aodh::params::listener_service_name, + enable => $enabled, + hasstatus => true, + hasrestart => true, + tag => 'aodh-service', + } +} diff --git a/aodh/manifests/notifier.pp b/aodh/manifests/notifier.pp new file mode 100644 index 000000000..f4629738b --- /dev/null +++ b/aodh/manifests/notifier.pp @@ -0,0 +1,49 @@ +# Installs the aodh notifier service +# +# == Params +# [*enabled*] +# (optional) Should the service be enabled. +# Defaults to true. +# +# [*manage_service*] +# (optional) Whether the service should be managed by Puppet. +# Defaults to true. +# +# [*package_ensure*] +# (optional) ensure state for package. +# Defaults to 'present' +# +class aodh::notifier ( + $manage_service = true, + $enabled = true, + $package_ensure = 'present', +) { + + include ::aodh::params + + Aodh_config<||> ~> Service['aodh-notifier'] + + Package[$::aodh::params::notifier_package_name] -> Service['aodh-notifier'] + ensure_resource( 'package', [$::aodh::params::notifier_package_name], + { ensure => $package_ensure, + tag => ['openstack', 'aodh-package'] } + ) + + if $manage_service { + if $enabled { + $service_ensure = 'running' + } else { + $service_ensure = 'stopped' + } + } + + Package['aodh'] -> Service['aodh-notifier'] + service { 'aodh-notifier': + ensure => $service_ensure, + name => $::aodh::params::notifier_service_name, + enable => $enabled, + hasstatus => true, + hasrestart => true, + tag => 'aodh-service', + } +} diff --git a/aodh/manifests/params.pp b/aodh/manifests/params.pp index 1b7d1575b..e769cf789 100644 --- a/aodh/manifests/params.pp +++ b/aodh/manifests/params.pp @@ -2,6 +2,8 @@ # class aodh::params { + $client_package_name = 'python-ceilometerclient' + case $::osfamily { 'RedHat': { $common_package_name = 'openstack-aodh-common' diff --git a/aodh/metadata.json b/aodh/metadata.json index 8e41b1553..3d56e7637 100644 --- a/aodh/metadata.json +++ b/aodh/metadata.json @@ -1,6 +1,6 @@ { "name": "puppet-aodh", - "version": "0.0.1", + "version": "7.0.0", "author": "OpenStack Contributors", "summary": "Puppet module for OpenStack Aodh", "license": "Apache-2.0", @@ -29,6 +29,6 @@ "dependencies": [ { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">= 4.2.0 <5.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" } + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" } ] } diff --git a/aodh/spec/acceptance/aodh_wsgi_apache_spec.rb b/aodh/spec/acceptance/aodh_wsgi_apache_spec.rb index 0fe2083db..12d29e5d7 100644 --- a/aodh/spec/acceptance/aodh_wsgi_apache_spec.rb +++ b/aodh/spec/acceptance/aodh_wsgi_apache_spec.rb @@ -6,59 +6,11 @@ it 'should work with no errors' do pp= <<-EOS - Exec { logoutput => 'on_failure' } - - # Common resources - case $::osfamily { - 'Debian': { - include ::apt - class { '::openstack_extras::repo::debian::ubuntu': - release => 'liberty', - repo => 'proposed', - package_require => true, - } - $package_provider = 'apt' - } - 'RedHat': { - class { '::openstack_extras::repo::redhat::redhat': - manage_rdo => false, - repo_hash => { - 'openstack-common-testing' => { - 'baseurl' => 'http://cbs.centos.org/repos/cloud7-openstack-common-testing/x86_64/os/', - 'descr' => 'openstack-common-testing', - 'gpgcheck' => 'no', - }, - 'openstack-liberty-testing' => { - 'baseurl' => 'http://cbs.centos.org/repos/cloud7-openstack-liberty-testing/x86_64/os/', - 'descr' => 'openstack-liberty-testing', - 'gpgcheck' => 'no', - }, - 'openstack-liberty-trunk' => { - 'baseurl' => 'http://trunk.rdoproject.org/centos7-liberty/current-passed-ci/', - 'descr' => 'openstack-liberty-trunk', - 'gpgcheck' => 'no', - }, - }, - } - package { 'openstack-selinux': ensure => 'latest' } - $package_provider = 'yum' - } - default: { - fail("Unsupported osfamily (${::osfamily})") - } - } - - class { '::mysql::server': } - - class { '::rabbitmq': - delete_guest_user => true, - package_provider => $package_provider, - } - - rabbitmq_vhost { '/': - provider => 'rabbitmqctl', - require => Class['rabbitmq'], - } + include ::openstack_integration + include ::openstack_integration::repos + include ::openstack_integration::rabbitmq + include ::openstack_integration::mysql + include ::openstack_integration::keystone rabbitmq_user { 'aodh': admin => true, @@ -75,26 +27,6 @@ class { '::rabbitmq': require => Class['rabbitmq'], } - - # Keystone resources, needed by Ceilometer to run - class { '::keystone::db::mysql': - password => 'keystone', - } - class { '::keystone': - verbose => true, - debug => true, - database_connection => 'mysql://keystone:keystone@127.0.0.1/keystone', - admin_token => 'admin_token', - enabled => true, - } - class { '::keystone::roles::admin': - email => 'test@example.tld', - password => 'a_big_secret', - } - class { '::keystone::endpoint': - public_url => "https://${::fqdn}:5000/", - admin_url => "https://${::fqdn}:35357/", - } class { '::aodh': rabbit_userid => 'aodh', rabbit_password => 'an_even_bigger_secret', @@ -119,6 +51,22 @@ class { '::aodh::api': class { '::aodh::wsgi::apache': ssl => false, } + class { '::aodh::auth': + auth_url => 'http://127.0.0.1:5000/v2.0', + auth_password => 'a_big_secret', + } + class { '::aodh::client': } + class { '::aodh::notifier': } + class { '::aodh::listener': } + case $::osfamily { + 'Debian': { + warning('aodh-evaluator cannot be run on ubuntu system, package is broken. See LP#1508463') + } + 'RedHat': { + class { '::aodh::evaluator': } + class { '::aodh::db::sync': } + } + } EOS diff --git a/aodh/spec/classes/aodh_api_spec.rb b/aodh/spec/classes/aodh_api_spec.rb index 284ccef40..3a8f032f3 100644 --- a/aodh/spec/classes/aodh_api_spec.rb +++ b/aodh/spec/classes/aodh_api_spec.rb @@ -14,7 +14,7 @@ :keystone_tenant => 'services', :keystone_user => 'aodh', :package_ensure => 'latest', - :port => '8777', + :port => '8042', :host => '0.0.0.0', } end diff --git a/aodh/spec/classes/aodh_auth_spec.rb b/aodh/spec/classes/aodh_auth_spec.rb new file mode 100644 index 000000000..9813451c3 --- /dev/null +++ b/aodh/spec/classes/aodh_auth_spec.rb @@ -0,0 +1,55 @@ +require 'spec_helper' + +describe 'aodh::auth' do + + let :params do + { :auth_url => 'http://localhost:5000/v2.0', + :auth_region => 'RegionOne', + :auth_user => 'aodh', + :auth_password => 'password', + :auth_tenant_name => 'services', + } + end + + shared_examples_for 'aodh-auth' do + + it 'configures authentication' do + is_expected.to contain_aodh_config('service_credentials/os_auth_url').with_value('http://localhost:5000/v2.0') + is_expected.to contain_aodh_config('service_credentials/os_region_name').with_value('RegionOne') + is_expected.to contain_aodh_config('service_credentials/os_username').with_value('aodh') + is_expected.to contain_aodh_config('service_credentials/os_password').with_value('password') + is_expected.to contain_aodh_config('service_credentials/os_password').with_value(params[:auth_password]).with_secret(true) + is_expected.to contain_aodh_config('service_credentials/os_tenant_name').with_value('services') + is_expected.to contain_aodh_config('service_credentials/os_cacert').with(:ensure => 'absent') + end + + context 'when overriding parameters' do + before do + params.merge!( + :auth_cacert => '/tmp/dummy.pem', + :auth_endpoint_type => 'internalURL', + ) + end + it { is_expected.to contain_aodh_config('service_credentials/os_cacert').with_value(params[:auth_cacert]) } + it { is_expected.to contain_aodh_config('service_credentials/os_endpoint_type').with_value(params[:auth_endpoint_type]) } + end + + end + + context 'on Debian platforms' do + let :facts do + { :osfamily => 'Debian' } + end + + it_configures 'aodh-auth' + end + + context 'on RedHat platforms' do + let :facts do + { :osfamily => 'RedHat' } + end + + it_configures 'aodh-auth' + end + +end diff --git a/aodh/spec/classes/aodh_client_spec.rb b/aodh/spec/classes/aodh_client_spec.rb new file mode 100644 index 000000000..8c43670f5 --- /dev/null +++ b/aodh/spec/classes/aodh_client_spec.rb @@ -0,0 +1,33 @@ +require 'spec_helper' + +describe 'aodh::client' do + + shared_examples_for 'aodh client' do + + it { is_expected.to contain_class('aodh::params') } + + it 'installs aodh client package' do + is_expected.to contain_package('python-ceilometerclient').with( + :ensure => 'present', + :name => 'python-ceilometerclient', + :tag => 'openstack', + ) + end + end + + context 'on Debian platforms' do + let :facts do + { :osfamily => 'Debian' } + end + + it_configures 'aodh client' + end + + context 'on RedHat platforms' do + let :facts do + { :osfamily => 'RedHat' } + end + + it_configures 'aodh client' + end +end diff --git a/aodh/spec/classes/aodh_db_spec.rb b/aodh/spec/classes/aodh_db_spec.rb new file mode 100644 index 000000000..2b0b343c1 --- /dev/null +++ b/aodh/spec/classes/aodh_db_spec.rb @@ -0,0 +1,96 @@ +require 'spec_helper' + +describe 'aodh::db' do + + shared_examples 'aodh::db' do + + context 'with default parameters' do + + it { is_expected.to contain_class('aodh::params') } + it { is_expected.to contain_aodh_config('database/connection').with_value('sqlite:////var/lib/aodh/aodh.sqlite') } + it { is_expected.to contain_aodh_config('database/idle_timeout').with_value('3600') } + it { is_expected.to contain_aodh_config('database/min_pool_size').with_value('1') } + it { is_expected.to contain_aodh_config('database/max_retries').with_value('10') } + it { is_expected.to contain_aodh_config('database/retry_interval').with_value('10') } + + end + + context 'with specific parameters' do + let :params do + { :database_connection => 'mysql://aodh:aodh@localhost/aodh', + :database_idle_timeout => '3601', + :database_min_pool_size => '2', + :database_max_retries => '11', + :database_retry_interval => '11', + } + end + + it { is_expected.to contain_class('aodh::params') } + it { is_expected.to contain_aodh_config('database/connection').with_value('mysql://aodh:aodh@localhost/aodh').with_secret(true) } + it { is_expected.to contain_aodh_config('database/idle_timeout').with_value('3601') } + it { is_expected.to contain_aodh_config('database/min_pool_size').with_value('2') } + it { is_expected.to contain_aodh_config('database/max_retries').with_value('11') } + it { is_expected.to contain_aodh_config('database/retry_interval').with_value('11') } + + end + + context 'with postgresql backend' do + let :params do + { :database_connection => 'postgresql://localhost:1234/aodh', } + end + + it 'install the proper backend package' do + is_expected.to contain_package('python-psycopg2').with(:ensure => 'present') + end + + end + + + context 'with incorrect database_connection string' do + let :params do + { :database_connection => 'redis://aodh:aodh@localhost/aodh', } + end + + it_raises 'a Puppet::Error', /validate_re/ + end + + end + + context 'on Debian platforms' do + let :facts do + { :osfamily => 'Debian', + :operatingsystem => 'Debian', + :operatingsystemrelease => 'jessie', + } + end + + it_configures 'aodh::db' + + context 'with sqlite backend' do + let :params do + { :database_connection => 'sqlite:///var/lib/aodh/aodh.sqlite', } + end + + it 'install the proper backend package' do + is_expected.to contain_package('aodh-backend-package').with( + :ensure => 'present', + :name => 'python-pysqlite2', + :tag => 'openstack' + ) + end + + end + end + + context 'on Redhat platforms' do + let :facts do + { :osfamily => 'RedHat', + :operatingsystemrelease => '7.1', + } + end + + it_configures 'aodh::db' + end + +end + diff --git a/aodh/spec/classes/aodh_evaluator_spec.rb b/aodh/spec/classes/aodh_evaluator_spec.rb new file mode 100644 index 000000000..7ada65f62 --- /dev/null +++ b/aodh/spec/classes/aodh_evaluator_spec.rb @@ -0,0 +1,113 @@ +require 'spec_helper' +# LP1492636 - Cohabitation of compile matcher and webmock +WebMock.disable_net_connect!(:allow => "169.254.169.254") + +describe 'aodh::evaluator' do + + let :pre_condition do + "class { '::aodh': }" + end + + let :params do + { :enabled => true } + end + + shared_examples_for 'aodh-evaluator' do + + context 'with coordination' do + before do + params.merge!({ :coordination_url => 'redis://localhost:6379' }) + end + + it 'configures backend_url' do + is_expected.to contain_aodh_config('coordination/backend_url').with_value('redis://localhost:6379') + end + end + + context 'when enabled' do + it { is_expected.to contain_class('aodh::params') } + + it 'installs aodh-evaluator package' do + is_expected.to contain_package(platform_params[:evaluator_package_name]).with( + :ensure => 'present', + :tag => ['openstack', 'aodh-package'] + ) + end + + it 'configures aodh-evaluator service' do + is_expected.to contain_service('aodh-evaluator').with( + :ensure => 'running', + :name => platform_params[:evaluator_service_name], + :enable => true, + :hasstatus => true, + :hasrestart => true, + :tag => ['aodh-service','aodh-db-sync-service'] + ) + end + + end + + context 'when disabled' do + let :params do + { :enabled => false } + end + + # Catalog compilation does not crash for lack of aodh::db + it { is_expected.to compile } + it 'configures aodh-evaluator service' do + is_expected.to contain_service('aodh-evaluator').with( + :ensure => 'stopped', + :name => platform_params[:evaluator_service_name], + :enable => false, + :hasstatus => true, + :hasrestart => true, + :tag => ['aodh-service','aodh-db-sync-service'] + ) + end + end + + context 'when service management is disabled' do + let :params do + { :enabled => false, + :manage_service => false } + end + + it 'configures aodh-evaluator service' do + is_expected.to contain_service('aodh-evaluator').with( + :ensure => nil, + :name => platform_params[:evaluator_service_name], + :enable => false, + :hasstatus => true, + :hasrestart => true, + :tag => ['aodh-service','aodh-db-sync-service'] + ) + end + end + end + + context 'on Debian platforms' do + let :facts do + { :osfamily => 'Debian' } + end + + let :platform_params do + { :evaluator_package_name => 'aodh-evaluator', + :evaluator_service_name => 'aodh-evaluator' } + end + + it_configures 'aodh-evaluator' + end + + context 'on RedHat platforms' do + let :facts do + { :osfamily => 'RedHat' } + end + + let :platform_params do + { :evaluator_package_name => 'openstack-aodh-evaluator', + :evaluator_service_name => 'openstack-aodh-evaluator' } + end + + it_configures 'aodh-evaluator' + end +end diff --git a/aodh/spec/classes/aodh_listener_spec.rb b/aodh/spec/classes/aodh_listener_spec.rb new file mode 100644 index 000000000..180b5f342 --- /dev/null +++ b/aodh/spec/classes/aodh_listener_spec.rb @@ -0,0 +1,99 @@ +require 'spec_helper' +# LP1492636 - Cohabitation of compile matcher and webmock +WebMock.disable_net_connect!(:allow => "169.254.169.254") + +describe 'aodh::listener' do + + let :pre_condition do + "class { '::aodh': }" + end + + shared_examples_for 'aodh-listener' do + + context 'when enabled' do + it { is_expected.to contain_class('aodh::params') } + + it 'installs aodh-listener package' do + is_expected.to contain_package(platform_params[:listener_package_name]).with( + :ensure => 'present', + :tag => ['openstack', 'aodh-package'] + ) + end + + it 'configures aodh-listener service' do + is_expected.to contain_service('aodh-listener').with( + :ensure => 'running', + :name => platform_params[:listener_service_name], + :enable => true, + :hasstatus => true, + :hasrestart => true, + :tag => 'aodh-service', + ) + end + + end + + context 'when disabled' do + let :params do + { :enabled => false } + end + + # Catalog compilation does not crash for lack of aodh::db + it { is_expected.to compile } + it 'configures aodh-listener service' do + is_expected.to contain_service('aodh-listener').with( + :ensure => 'stopped', + :name => platform_params[:listener_service_name], + :enable => false, + :hasstatus => true, + :hasrestart => true, + :tag => 'aodh-service', + ) + end + end + + context 'when service management is disabled' do + let :params do + { :enabled => false, + :manage_service => false } + end + + it 'configures aodh-listener service' do + is_expected.to contain_service('aodh-listener').with( + :ensure => nil, + :name => platform_params[:listener_service_name], + :enable => false, + :hasstatus => true, + :hasrestart => true, + :tag => 'aodh-service', + ) + end + end + end + + context 'on Debian platforms' do + let :facts do + { :osfamily => 'Debian' } + end + + let :platform_params do + { :listener_package_name => 'aodh-listener', + :listener_service_name => 'aodh-listener' } + end + + it_configures 'aodh-listener' + end + + context 'on RedHat platforms' do + let :facts do + { :osfamily => 'RedHat' } + end + + let :platform_params do + { :listener_package_name => 'openstack-aodh-listener', + :listener_service_name => 'openstack-aodh-listener' } + end + + it_configures 'aodh-listener' + end +end diff --git a/aodh/spec/classes/aodh_notifier_spec.rb b/aodh/spec/classes/aodh_notifier_spec.rb new file mode 100644 index 000000000..94d7b9cce --- /dev/null +++ b/aodh/spec/classes/aodh_notifier_spec.rb @@ -0,0 +1,99 @@ +require 'spec_helper' +# LP1492636 - Cohabitation of compile matcher and webmock +WebMock.disable_net_connect!(:allow => "169.254.169.254") + +describe 'aodh::notifier' do + + let :pre_condition do + "class { '::aodh': }" + end + + shared_examples_for 'aodh-notifier' do + + context 'when enabled' do + it { is_expected.to contain_class('aodh::params') } + + it 'installs aodh-notifier package' do + is_expected.to contain_package(platform_params[:notifier_package_name]).with( + :ensure => 'present', + :tag => ['openstack', 'aodh-package'] + ) + end + + it 'configures aodh-notifier service' do + is_expected.to contain_service('aodh-notifier').with( + :ensure => 'running', + :name => platform_params[:notifier_service_name], + :enable => true, + :hasstatus => true, + :hasrestart => true, + :tag => 'aodh-service', + ) + end + + end + + context 'when disabled' do + let :params do + { :enabled => false } + end + + # Catalog compilation does not crash for lack of aodh::db + it { is_expected.to compile } + it 'configures aodh-notifier service' do + is_expected.to contain_service('aodh-notifier').with( + :ensure => 'stopped', + :name => platform_params[:notifier_service_name], + :enable => false, + :hasstatus => true, + :hasrestart => true, + :tag => 'aodh-service', + ) + end + end + + context 'when service management is disabled' do + let :params do + { :enabled => false, + :manage_service => false } + end + + it 'configures aodh-notifier service' do + is_expected.to contain_service('aodh-notifier').with( + :ensure => nil, + :name => platform_params[:notifier_service_name], + :enable => false, + :hasstatus => true, + :hasrestart => true, + :tag => 'aodh-service', + ) + end + end + end + + context 'on Debian platforms' do + let :facts do + { :osfamily => 'Debian' } + end + + let :platform_params do + { :notifier_package_name => 'aodh-notifier', + :notifier_service_name => 'aodh-notifier' } + end + + it_configures 'aodh-notifier' + end + + context 'on RedHat platforms' do + let :facts do + { :osfamily => 'RedHat' } + end + + let :platform_params do + { :notifier_package_name => 'openstack-aodh-notifier', + :notifier_service_name => 'openstack-aodh-notifier' } + end + + it_configures 'aodh-notifier' + end +end diff --git a/aodh/spec/spec_helper_acceptance.rb b/aodh/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/aodh/spec/spec_helper_acceptance.rb +++ b/aodh/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/ceilometer/CHANGELOG.md b/ceilometer/CHANGELOG.md index 0dc1407d8..461000390 100644 --- a/ceilometer/CHANGELOG.md +++ b/ceilometer/CHANGELOG.md @@ -1,3 +1,50 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes +- change section name for AMQP qpid parameters +- remove deprecated mysql_module + +####Features +- keystone/auth: make service description configurable +- add support for RabbitMQ connection heartbeat +- simplify parameters for rpc_backend +- add tag to package and service resources +- enable support for memcached_servers +- add ability to specify ttl and timeout parameters +- add ability to manage use_stderr parameter +- creation of ceilometer::db::sync +- reflect provider change in puppet-openstacklib +- make 'alarm_history_time_to_live' parameter configurable +- update ceilometer::db class to match other module pattern +- implement auth_endpoint_type parameter +- stop managing File resources +- put all the logging related parameters to the logging class +- add mongodb_replica_set option +- allow customization of db sync command line + +####Bugfixes +- rely on autorequire for config resource ordering +- compute agent: do not try to configure nova.conf +- agent/auth: bring consistent how we manage empty parameters +- remove the api service subscription on db sync +- wsgi: make sure eventlet process is stopped before httpd +- auth: drop service dependency for Keystone_user_role + +####Maintenance +- fix rspec 3.x syntax +- initial msync run for all Puppet OpenStack modules +- acceptance: enable debug & verbosity for OpenStack logs +- acceptance/eventlet: make sure apache is stopped +- acceptance: use common bits from puppet-openstack-integration +- rspec: run tests for ::ceilometer::agent::auth +- try to use zuul-cloner to prepare fixtures +- spec: Enable webmock connect to IPv4 link-local +- db: Use postgresql lib class for psycopg package +- remove class_parameter_defaults puppet-lint check + ##2015-10-10 - 6.1.0 ###Summary diff --git a/ceilometer/README.md b/ceilometer/README.md index 8c0b40719..b9cf4b298 100644 --- a/ceilometer/README.md +++ b/ceilometer/README.md @@ -1,7 +1,7 @@ Ceilometer ========== -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2 - Liberty #### Table of Contents diff --git a/ceilometer/Rakefile b/ceilometer/Rakefile index ed79bead4..9692ffdd2 100644 --- a/ceilometer/Rakefile +++ b/ceilometer/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/ceilometer/manifests/init.pp b/ceilometer/manifests/init.pp index 2cfdf0910..2200d7738 100644 --- a/ceilometer/manifests/init.pp +++ b/ceilometer/manifests/init.pp @@ -263,6 +263,8 @@ # we keep "ceilometer.openstack.common.rpc.impl_qpid" for backward compatibility if $rpc_backend == 'ceilometer.openstack.common.rpc.impl_qpid' or $rpc_backend == 'qpid' { + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + ceilometer_config { 'oslo_messaging_qpid/qpid_hostname' : value => $qpid_hostname; 'oslo_messaging_qpid/qpid_port' : value => $qpid_port; diff --git a/ceilometer/metadata.json b/ceilometer/metadata.json index 207c7d5a7..c7fc20a4b 100644 --- a/ceilometer/metadata.json +++ b/ceilometer/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-ceilometer", - "version": "6.1.0", + "version": "7.0.0", "author": "eNovance and OpenStack Contributors", "summary": "Puppet module for OpenStack Ceilometer", "license": "Apache-2.0", @@ -33,8 +33,8 @@ "dependencies": [ { "name": "puppetlabs/apache", "version_requirement": ">=1.0.0 <2.0.0" }, { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" } + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" } ] } diff --git a/ceilometer/spec/classes/ceilometer_api_spec.rb b/ceilometer/spec/classes/ceilometer_api_spec.rb index 094e2181b..7aefed07c 100644 --- a/ceilometer/spec/classes/ceilometer_api_spec.rb +++ b/ceilometer/spec/classes/ceilometer_api_spec.rb @@ -172,12 +172,13 @@ class { 'ceilometer': metering_secret => 's3cr3t' }" context 'on RedHat platforms' do let :facts do - { :osfamily => 'RedHat', - :operatingsystem => 'RedHat', - :operatingsystemrelease => '7.1', - :fqdn => 'some.host.tld', - :concat_basedir => '/var/lib/puppet/concat', - :processorcount => 2 } + { :osfamily => 'RedHat', + :operatingsystem => 'RedHat', + :operatingsystemrelease => '7.1', + :operatingsystemmajrelease => '7', + :fqdn => 'some.host.tld', + :concat_basedir => '/var/lib/puppet/concat', + :processorcount => 2 } end let :platform_params do diff --git a/ceilometer/spec/spec_helper_acceptance.rb b/ceilometer/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/ceilometer/spec/spec_helper_acceptance.rb +++ b/ceilometer/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/cinder/CHANGELOG.md b/cinder/CHANGELOG.md index a741ab0bb..da520ebbb 100644 --- a/cinder/CHANGELOG.md +++ b/cinder/CHANGELOG.md @@ -1,3 +1,55 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes +- rabbitmq: do not manage rabbitmq service anymore +- remove deprecated mysql_module +- change section name for AMQP qpid parameters +- change section name for AMQP rabbit parameters + +####Features +- make it possible to have multiple type_sets with the same value +- keystone/auth: make service description configurable +- add support for RabbitMQ connection heartbeat +- add tag to package and service resources +- sync cinder::db::sync with new standard +- set parameter host in vmdk backend +- add lock_path to cinder config +- add 'host' parameter to cinder.conf +- add an ability to manage use_stderr parameter +- reflect provider change in puppet-openstacklib +- introduce cinder::db class +- add support for Pure Volume Drivers +- implement class and defined type for cinder GPFS driver +- allow RBD host to be configurable +- add posibility to specify privileged user for Cinder +- switch cinder to use os_service_default +- simplify rpc_backend parameter +- put all the logging related parameters to the logging class +- specify a dedicated keystone user for cinderv2 +- db: Use postgresql lib class for psycopg package +- add kombu_reconnect_delay parameter to cinder +- allow customization of db sync command line +- add possibility to use list of qpid hosts in cinder + +####Bugfixes +- rely on autorequire for config resource ordering +- dont run losetup if it's been done already +- don't run vgcreate if vg exists +- make sure cinder user is created before creating Cinder::Type +- type: Add retry to Exec resource + +####Maintenance +- fix rspec 3.x syntax +- initial msync run for all Puppet OpenStack modules +- acceptance: enable debug & verbosity for OpenStack logs +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- add test coverage for cinder::db::mysql +- acceptance: use common bits from puppet-openstack-integration + ##2015-10-10 - 6.1.0 ###Summary diff --git a/cinder/README.md b/cinder/README.md index af609d1f3..cff85d8f2 100644 --- a/cinder/README.md +++ b/cinder/README.md @@ -1,7 +1,7 @@ cinder ======= -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2 - Liberty #### Table of Contents diff --git a/cinder/Rakefile b/cinder/Rakefile index ed79bead4..9692ffdd2 100644 --- a/cinder/Rakefile +++ b/cinder/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/cinder/manifests/api.pp b/cinder/manifests/api.pp index 6ffd8a6f3..0155dccb5 100644 --- a/cinder/manifests/api.pp +++ b/cinder/manifests/api.pp @@ -56,6 +56,11 @@ # (optional) Auth URL associated with the OpenStack privileged account. # Defaults to $::os_service_default. # +# [*keymgr_encryption_auth_url*] +# (optional) Auth URL for keymgr authentication. Should be in format +# http://auth_url:5000/v3 +# Defaults to $::os_service_default. +# # [*os_region_name*] # (optional) Some operations require cinder to make API requests # to Nova. This sets the keystone region to be used for these @@ -169,6 +174,7 @@ $os_privileged_user_password = $::os_service_default, $os_privileged_user_tenant = $::os_service_default, $os_privileged_user_auth_url = $::os_service_default, + $keymgr_encryption_auth_url = $::os_service_default, $service_workers = $::processorcount, $package_ensure = 'present', $bind_host = '0.0.0.0', @@ -277,70 +283,56 @@ else { $auth_uri_real = $auth_uri } - cinder_api_paste_ini { 'filter:authtoken/auth_uri': value => $auth_uri_real; } + + cinder_config { + 'keystone_authtoken/auth_uri': value => $auth_uri_real; + 'keymgr/encryption_auth_url' : value => $keymgr_encryption_auth_url; + } if $keystone_enabled { cinder_config { - 'DEFAULT/auth_strategy': value => 'keystone' ; + 'DEFAULT/auth_strategy': value => 'keystone' ; + 'keystone_authtoken/admin_tenant_name': value => $keystone_tenant; + 'keystone_authtoken/admin_user': value => $keystone_user; + 'keystone_authtoken/admin_password': value => $keystone_password, secret => true; } - cinder_api_paste_ini { - 'filter:authtoken/admin_tenant_name': value => $keystone_tenant; - 'filter:authtoken/admin_user': value => $keystone_user; - 'filter:authtoken/admin_password': value => $keystone_password, secret => true; - } # if both auth_uri and identity_uri are set we skip these deprecated settings entirely if !$auth_uri or !$identity_uri { if $keystone_auth_host { warning('The keystone_auth_host parameter is deprecated. Please use auth_uri and identity_uri instead.') - cinder_api_paste_ini { - 'filter:authtoken/service_host': value => $keystone_auth_host; - 'filter:authtoken/auth_host': value => $keystone_auth_host; + cinder_config { + 'keystone_authtoken/auth_host': value => $keystone_auth_host; } } else { - cinder_api_paste_ini { - 'filter:authtoken/service_host': ensure => absent; - 'filter:authtoken/auth_host': ensure => absent; + cinder_config { + 'keystone_authtoken/auth_host': ensure => absent; } } if $keystone_auth_protocol { warning('The keystone_auth_protocol parameter is deprecated. Please use auth_uri and identity_uri instead.') - cinder_api_paste_ini { - 'filter:authtoken/service_protocol': value => $keystone_auth_protocol; - 'filter:authtoken/auth_protocol': value => $keystone_auth_protocol; + cinder_config { + 'keystone_authtoken/auth_protocol': value => $keystone_auth_protocol; } } else { - cinder_api_paste_ini { - 'filter:authtoken/service_protocol': ensure => absent; - 'filter:authtoken/auth_protocol': ensure => absent; + cinder_config { + 'keystone_authtoken/auth_protocol': ensure => absent; } } if $keystone_auth_port { warning('The keystone_auth_port parameter is deprecated. Please use auth_uri and identity_uri instead.') - cinder_api_paste_ini { - 'filter:authtoken/auth_port': value => $keystone_auth_port; + cinder_config { + 'keystone_authtoken/auth_port': value => $keystone_auth_port; } } else { - cinder_api_paste_ini { - 'filter:authtoken/auth_port': ensure => absent; + cinder_config { + 'keystone_authtoken/auth_port': ensure => absent; } } - if $service_port { - warning('The service_port parameter is deprecated. Please use auth_uri and identity_uri instead.') - cinder_api_paste_ini { - 'filter:authtoken/service_port': value => $service_port; - } - } else { - cinder_api_paste_ini { - 'filter:authtoken/service_port': ensure => absent; - } - } - - if $keystone_auth_admin_prefix { warning('The keystone_auth_admin_prefix parameter is deprecated. Please use auth_uri and identity_uri instead.') validate_re($keystone_auth_admin_prefix, '^(/.+[^/])?$') @@ -357,24 +349,21 @@ cinder_api_paste_ini { 'filter:authtoken/auth_admin_prefix': ensure => absent; } - cinder_api_paste_ini { - 'filter:authtoken/service_port': ensure => absent; - 'filter:authtoken/auth_port': ensure => absent; - 'filter:authtoken/service_host': ensure => absent; - 'filter:authtoken/auth_host': ensure => absent; - 'filter:authtoken/service_protocol': ensure => absent; - 'filter:authtoken/auth_protocol': ensure => absent; + cinder_config { + 'keystone_authtoken/auth_port': ensure => absent; + 'keystone_authtoken/auth_host': ensure => absent; + 'keystone_authtoken/auth_protocol': ensure => absent; } } + } - if $identity_uri { - cinder_api_paste_ini { - 'filter:authtoken/identity_uri': value => $identity_uri; - } - } else { - cinder_api_paste_ini { - 'filter:authtoken/identity_uri': ensure => absent; - } + if $identity_uri { + cinder_config { + 'keystone_authtoken/identity_uri': value => $identity_uri; + } + } else { + cinder_config { + 'keystone_authtoken/identity_uri': ensure => absent; } } diff --git a/cinder/manifests/init.pp b/cinder/manifests/init.pp index 863056643..ac8056e0d 100644 --- a/cinder/manifests/init.pp +++ b/cinder/manifests/init.pp @@ -357,6 +357,8 @@ if $rpc_backend == 'cinder.openstack.common.rpc.impl_qpid' or $rpc_backend == 'qpid' { + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + if ! $qpid_password { fail('Please specify a qpid_password parameter.') } diff --git a/cinder/manifests/qpid.pp b/cinder/manifests/qpid.pp index b5eab2a2b..307e1e584 100644 --- a/cinder/manifests/qpid.pp +++ b/cinder/manifests/qpid.pp @@ -33,6 +33,8 @@ $realm ='OPENSTACK' ) { + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + # only configure cinder after the queue is up Class['qpid::server'] -> Package<| title == 'cinder' |> diff --git a/cinder/metadata.json b/cinder/metadata.json index 5842eca84..3253b2e3c 100644 --- a/cinder/metadata.json +++ b/cinder/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-cinder", - "version": "6.1.0", + "version": "7.0.0", "author": "Puppet Labs and OpenStack Contributors", "summary": "Puppet module for OpenStack Cinder", "license": "Apache-2.0", @@ -33,9 +33,9 @@ "dependencies": [ { "name": "dprince/qpid", "version_requirement": ">=1.0.0 <2.0.0" }, { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/rabbitmq", "version_requirement": ">=2.0.2 <6.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" } + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" } ] } diff --git a/cinder/spec/classes/cinder_api_spec.rb b/cinder/spec/classes/cinder_api_spec.rb index 15bcba38a..20ba39640 100644 --- a/cinder/spec/classes/cinder_api_spec.rb +++ b/cinder/spec/classes/cinder_api_spec.rb @@ -45,45 +45,45 @@ is_expected.to contain_cinder_config('DEFAULT/os_region_name').with( :value => '' ) - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/service_protocol').with( - :value => 'http' + is_expected.to contain_cinder_config('keystone_authtoken/auth_uri').with( + :value => 'http://localhost:5000/' ) - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/service_host').with( - :value => 'localhost' + is_expected.to contain_cinder_config('keystone_authtoken/admin_tenant_name').with( + :value => 'services' ) - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/service_port').with( - :value => '5000' + is_expected.to contain_cinder_config('keystone_authtoken/admin_user').with( + :value => 'cinder' ) - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_protocol').with( + is_expected.to contain_cinder_config('keystone_authtoken/admin_password').with( + :value => 'foo' + ) + is_expected.to contain_cinder_config('keystone_authtoken/auth_protocol').with( :value => 'http' ) - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_host').with( + is_expected.to contain_cinder_config('keystone_authtoken/auth_host').with( :value => 'localhost' ) - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_port').with( + is_expected.to contain_cinder_config('keystone_authtoken/auth_port').with( :value => '35357' ) - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_admin_prefix').with( - :ensure => 'absent' - ) - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/admin_tenant_name').with( - :value => 'services' + is_expected.not_to contain_cinder_config('keystone_authtoken/service_protocol').with( + :value => 'http' ) - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/admin_user').with( - :value => 'cinder' + is_expected.not_to contain_cinder_config('keystone_authtoken/service_host').with( + :value => 'localhost' ) - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/admin_password').with( - :value => 'foo', - :secret => true + is_expected.not_to contain_cinder_config('keystone_authtoken/service_port').with( + :value => '5000' ) - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_uri').with( - :value => 'http://localhost:5000/' + is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_admin_prefix').with( + :ensure => 'absent' ) is_expected.to contain_cinder_config('DEFAULT/os_privileged_user_name').with_value('') is_expected.to contain_cinder_config('DEFAULT/os_privileged_user_password').with_value('') is_expected.to contain_cinder_config('DEFAULT/os_privileged_user_tenant').with_value('') is_expected.to contain_cinder_config('DEFAULT/os_privileged_user_auth_url').with_value('') + is_expected.to contain_cinder_config('keymgr/encryption_auth_url').with_value('') end end @@ -194,7 +194,7 @@ req_params.merge({'keystone_auth_uri' => 'http://localhost:8080/v2.0/'}) end it 'should configure cinder auth_uri correctly' do - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_uri').with( + is_expected.to contain_cinder_config('keystone_authtoken/auth_uri').with( :value => 'http://localhost:8080/v2.0/' ) end @@ -303,6 +303,16 @@ )} end + describe 'with encryption_auth_url' do + let :params do + req_params.merge({ :keymgr_encryption_auth_url => 'http://localhost:5000/v3' }) + end + + it { is_expected.to contain_cinder_config('keymgr/encryption_auth_url').with( + :value => 'http://localhost:5000/v3' + )} + end + describe 'while validating the service with default command' do let :params do req_params.merge({ @@ -350,15 +360,15 @@ }) end it 'configures identity_uri and auth_uri but deprecates old auth settings' do - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/identity_uri').with_value("https://localhost:35357/"); - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_uri').with_value("https://localhost:5000/v2.0/"); + is_expected.to contain_cinder_config('keystone_authtoken/identity_uri').with_value("https://localhost:35357/") + is_expected.to contain_cinder_config('keystone_authtoken/auth_uri').with_value("https://localhost:5000/v2.0/") is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_admin_prefix').with(:ensure => 'absent') - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_port').with(:ensure => 'absent') - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/service_port').with(:ensure => 'absent') - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_protocol').with(:ensure => 'absent') - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/service_protocol').with(:ensure => 'absent') - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/auth_host').with(:ensure => 'absent') - is_expected.to contain_cinder_api_paste_ini('filter:authtoken/service_host').with(:ensure => 'absent') + is_expected.to contain_cinder_config('keystone_authtoken/auth_port').with(:ensure => 'absent') + is_expected.not_to contain_cinder_config('keystone_authtoken/service_port').with(:ensure => 'absent') + is_expected.to contain_cinder_config('keystone_authtoken/auth_protocol').with(:ensure => 'absent') + is_expected.not_to contain_cinder_config('keystone_authtoken/service_protocol').with(:ensure => 'absent') + is_expected.to contain_cinder_config('keystone_authtoken/auth_host').with(:ensure => 'absent') + is_expected.not_to contain_cinder_config('keystone_authtoken/service_host').with(:ensure => 'absent') end end diff --git a/cinder/spec/spec_helper_acceptance.rb b/cinder/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/cinder/spec/spec_helper_acceptance.rb +++ b/cinder/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/glance/CHANGELOG.md b/glance/CHANGELOG.md index c168463d4..33d78f1c5 100644 --- a/glance/CHANGELOG.md +++ b/glance/CHANGELOG.md @@ -1,3 +1,44 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes +- remove deprecated mysql_module +- change section name for AMQP qpid parameters +- change section name for AMQP rabbit parameters + +####Features +- add support for RabbitMQ connection heartbeat +- add tag to package and service resources +- add glance::db::sync +- add an ability to manage use_stderr parameter +- reflect provider change in puppet-openstacklib +- put all the logging related parameters to the logging class +- allow customization of db sync command line +- add S3 backend configuration for glance +- add rados_connect_timeout parameter in glance config +- add ability to specify number of workers for glance-registry service +- use OpenstackClient for glance_image auth + +####Bugfixes +- rely on autorequire for config resource ordering +- make sure Facter is only executed on agent +- file backend: do not inherit from glance::api +- glance_image: hardcode os-image-api-version to 1 +- make sure Glance_image is executed after Keystone_endpoint +- solve duplicate declaration issue for python-openstackclient +- append openstacklib/lib to load path for type + +####Maintenance +- fix rspec 3.x syntax +- initial msync run for all Puppet OpenStack modules +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- acceptance: use common bits from puppet-openstack-integration +- fix unit tests against Puppet 4.3.0 +- require at least 4.2.0 of stdlib + ##2015-10-10 - 6.1.0 ###Summary diff --git a/glance/README.md b/glance/README.md index 0ac189459..376b19ac8 100644 --- a/glance/README.md +++ b/glance/README.md @@ -1,7 +1,7 @@ glance ======= -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2 - Liberty #### Table of Contents diff --git a/glance/Rakefile b/glance/Rakefile index ed79bead4..9692ffdd2 100644 --- a/glance/Rakefile +++ b/glance/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/glance/manifests/backend/swift.pp b/glance/manifests/backend/swift.pp index ea3a34b57..56f58051b 100644 --- a/glance/manifests/backend/swift.pp +++ b/glance/manifests/backend/swift.pp @@ -32,6 +32,12 @@ # == class: glance::backend::swift # [*swift_store_region*] # Optional. Default: undef # +# [*default_swift_reference*] +# Optional. The reference to the default swift +# account/backing store parameters to use for adding +# new images. String value. +# Default to 'ref1'. +# class glance::backend::swift( $swift_store_user, $swift_store_key, @@ -42,35 +48,26 @@ # == class: glance::backend::swift $swift_store_create_container_on_put = false, $swift_store_endpoint_type = 'internalURL', $swift_store_region = undef, + $default_swift_reference = 'ref1', ) { glance_api_config { - 'glance_store/default_store': value => 'swift'; - 'glance_store/swift_store_user': value => $swift_store_user; - 'glance_store/swift_store_key': value => $swift_store_key; - 'glance_store/swift_store_auth_address': value => $swift_store_auth_address; - 'glance_store/swift_store_region': value => $swift_store_region; - 'glance_store/swift_store_container': value => $swift_store_container; - 'glance_store/swift_store_auth_version': value => $swift_store_auth_version; + 'glance_store/default_store': value => 'swift'; + 'glance_store/swift_store_region': value => $swift_store_region; + 'glance_store/swift_store_container': value => $swift_store_container; 'glance_store/swift_store_create_container_on_put': value => $swift_store_create_container_on_put; 'glance_store/swift_store_large_object_size': value => $swift_store_large_object_size; 'glance_store/swift_store_endpoint_type': value => $swift_store_endpoint_type; - } - glance_cache_config { - 'glance_store/swift_store_user': value => $swift_store_user; - 'glance_store/swift_store_key': value => $swift_store_key; - 'glance_store/swift_store_auth_address': value => $swift_store_auth_address; - 'glance_store/swift_store_region': value => $swift_store_region; - 'glance_store/swift_store_container': value => $swift_store_container; - 'glance_store/swift_store_auth_version': value => $swift_store_auth_version; - 'glance_store/swift_store_create_container_on_put': - value => $swift_store_create_container_on_put; - 'glance_store/swift_store_large_object_size': - value => $swift_store_large_object_size; + 'DEFAULT/swift_store_config_file': value => '/etc/glance/glance-api.conf'; + 'glance_store/default_swift_reference': value => $default_swift_reference; + "${default_swift_reference}/user": value => $swift_store_user; + "${default_swift_reference}/key": value => $swift_store_key; + "${default_swift_reference}/auth_address": value => $swift_store_auth_address; + "${default_swift_reference}/auth_version": value => $swift_store_auth_version; } } diff --git a/glance/manifests/notify/qpid.pp b/glance/manifests/notify/qpid.pp index 7d12e162b..c3534c567 100644 --- a/glance/manifests/notify/qpid.pp +++ b/glance/manifests/notify/qpid.pp @@ -31,6 +31,8 @@ $qpid_protocol = 'tcp' ) inherits glance::api { + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + glance_api_config { 'DEFAULT/notifier_driver': value => 'qpid'; 'oslo_messaging_qpid/qpid_hostname': value => $qpid_hostname; diff --git a/glance/metadata.json b/glance/metadata.json index 787b828df..28e073b62 100644 --- a/glance/metadata.json +++ b/glance/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-glance", - "version": "6.1.0", + "version": "7.0.0", "author": "Puppet Labs and OpenStack Contributors", "summary": "Puppet module for OpenStack Glance", "license": "Apache-2.0", @@ -32,8 +32,8 @@ "description": "Installs and configures OpenStack Glance (Image Service).", "dependencies": [ { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.2.0 <5.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" } + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" } ] } diff --git a/glance/spec/classes/glance_backend_swift_spec.rb b/glance/spec/classes/glance_backend_swift_spec.rb index 0e561ff80..3922cfc6e 100644 --- a/glance/spec/classes/glance_backend_swift_spec.rb +++ b/glance/spec/classes/glance_backend_swift_spec.rb @@ -22,61 +22,49 @@ it 'configures glance-api.conf' do is_expected.to contain_glance_api_config('glance_store/default_store').with_value('swift') - is_expected.to contain_glance_api_config('glance_store/swift_store_key').with_value('key') - is_expected.to contain_glance_api_config('glance_store/swift_store_user').with_value('user') - is_expected.to contain_glance_api_config('glance_store/swift_store_auth_version').with_value('2') is_expected.to contain_glance_api_config('glance_store/swift_store_large_object_size').with_value('5120') - is_expected.to contain_glance_api_config('glance_store/swift_store_auth_address').with_value('127.0.0.1:5000/v2.0/') is_expected.to contain_glance_api_config('glance_store/swift_store_container').with_value('glance') is_expected.to contain_glance_api_config('glance_store/swift_store_create_container_on_put').with_value(false) is_expected.to contain_glance_api_config('glance_store/swift_store_endpoint_type').with_value('internalURL') is_expected.to contain_glance_api_config('glance_store/swift_store_region').with_value(nil) + is_expected.to contain_glance_api_config('DEFAULT/swift_store_config_file').with_value('/etc/glance/glance-api.conf') + is_expected.to contain_glance_api_config('glance_store/default_swift_reference').with_value('ref1') + is_expected.to contain_glance_api_config('ref1/key').with_value('key') + is_expected.to contain_glance_api_config('ref1/user').with_value('user') + is_expected.to contain_glance_api_config('ref1/auth_version').with_value('2') + is_expected.to contain_glance_api_config('ref1/auth_address').with_value('127.0.0.1:5000/v2.0/') end - it 'configures glance-cache.conf' do - is_expected.to contain_glance_cache_config('glance_store/swift_store_key').with_value('key') - is_expected.to contain_glance_cache_config('glance_store/swift_store_user').with_value('user') - is_expected.to contain_glance_cache_config('glance_store/swift_store_auth_version').with_value('2') - is_expected.to contain_glance_cache_config('glance_store/swift_store_large_object_size').with_value('5120') - is_expected.to contain_glance_cache_config('glance_store/swift_store_auth_address').with_value('127.0.0.1:5000/v2.0/') - is_expected.to contain_glance_cache_config('glance_store/swift_store_container').with_value('glance') - is_expected.to contain_glance_cache_config('glance_store/swift_store_create_container_on_put').with_value(false) - is_expected.to contain_glance_cache_config('glance_store/swift_store_region').with_value(nil) - end end describe 'when overriding parameters' do let :params do { - :swift_store_user => 'user', - :swift_store_key => 'key', + :swift_store_user => 'user2', + :swift_store_key => 'key2', :swift_store_auth_version => '1', :swift_store_large_object_size => '100', :swift_store_auth_address => '127.0.0.2:8080/v1.0/', :swift_store_container => 'swift', :swift_store_create_container_on_put => true, :swift_store_endpoint_type => 'publicURL', - :swift_store_region => 'RegionTwo' + :swift_store_region => 'RegionTwo', + :default_swift_reference => 'swift_creds', } end it 'configures glance-api.conf' do is_expected.to contain_glance_api_config('glance_store/swift_store_container').with_value('swift') is_expected.to contain_glance_api_config('glance_store/swift_store_create_container_on_put').with_value(true) - is_expected.to contain_glance_api_config('glance_store/swift_store_auth_version').with_value('1') is_expected.to contain_glance_api_config('glance_store/swift_store_large_object_size').with_value('100') - is_expected.to contain_glance_api_config('glance_store/swift_store_auth_address').with_value('127.0.0.2:8080/v1.0/') is_expected.to contain_glance_api_config('glance_store/swift_store_endpoint_type').with_value('publicURL') is_expected.to contain_glance_api_config('glance_store/swift_store_region').with_value('RegionTwo') + is_expected.to contain_glance_api_config('glance_store/default_swift_reference').with_value('swift_creds') + is_expected.to contain_glance_api_config('swift_creds/key').with_value('key2') + is_expected.to contain_glance_api_config('swift_creds/user').with_value('user2') + is_expected.to contain_glance_api_config('swift_creds/auth_version').with_value('1') + is_expected.to contain_glance_api_config('swift_creds/auth_address').with_value('127.0.0.2:8080/v1.0/') end - it 'configures glance-cache.conf' do - is_expected.to contain_glance_cache_config('glance_store/swift_store_container').with_value('swift') - is_expected.to contain_glance_cache_config('glance_store/swift_store_create_container_on_put').with_value(true) - is_expected.to contain_glance_cache_config('glance_store/swift_store_auth_version').with_value('1') - is_expected.to contain_glance_cache_config('glance_store/swift_store_large_object_size').with_value('100') - is_expected.to contain_glance_cache_config('glance_store/swift_store_auth_address').with_value('127.0.0.2:8080/v1.0/') - is_expected.to contain_glance_cache_config('glance_store/swift_store_region').with_value('RegionTwo') - end end end diff --git a/glance/spec/spec_helper_acceptance.rb b/glance/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/glance/spec/spec_helper_acceptance.rb +++ b/glance/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/gnocchi/CHANGELOG.md b/gnocchi/CHANGELOG.md new file mode 100644 index 000000000..9db850184 --- /dev/null +++ b/gnocchi/CHANGELOG.md @@ -0,0 +1,4 @@ +##2015-11-25 - 7.0.0 +###Summary + +- Initial release of the puppet-gnocchi module diff --git a/gnocchi/README.md b/gnocchi/README.md index 00ac0ed86..30a3a78e4 100644 --- a/gnocchi/README.md +++ b/gnocchi/README.md @@ -1,6 +1,8 @@ puppet-gnocchi ============== +7.0.0 - 2015.2.0 - Liberty + #### Table of Contents 1. [Overview - What is the gnocchi module?](#overview) diff --git a/gnocchi/Rakefile b/gnocchi/Rakefile index ed79bead4..9692ffdd2 100644 --- a/gnocchi/Rakefile +++ b/gnocchi/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/gnocchi/examples/site.pp b/gnocchi/examples/site.pp index 0b9ded95c..2598a40ea 100644 --- a/gnocchi/examples/site.pp +++ b/gnocchi/examples/site.pp @@ -17,3 +17,11 @@ identity_uri => 'https://identity.openstack.org:35357', keystone_password => 'verysecrete' } + +class { '::gnocchi::statsd': + resource_id => '07f26121-5777-48ba-8a0b-d70468133dd9', + user_id => 'f81e9b1f-9505-4298-bc33-43dfbd9a973b', + project_id => '203ef419-e73f-4b8a-a73f-3d599a72b18d', + archive_policy_name => 'high', + flush_delay => '100', +} diff --git a/gnocchi/manifests/api.pp b/gnocchi/manifests/api.pp index 9c6ced6dc..be019fcc4 100644 --- a/gnocchi/manifests/api.pp +++ b/gnocchi/manifests/api.pp @@ -76,7 +76,6 @@ $service_name = $::gnocchi::params::api_service_name, ) inherits gnocchi::params { - include ::gnocchi::params include ::gnocchi::policy validate_string($keystone_password) @@ -120,6 +119,7 @@ tag => ['gnocchi-service', 'gnocchi-db-sync-service'], } Class['gnocchi::db'] -> Service[$service_name] + Service <<| title == 'httpd' |>> { tag +> 'gnocchi-db-sync-service' } # we need to make sure gnocchi-api/eventlet is stopped before trying to start apache Service['gnocchi-api'] -> Service[$service_name] diff --git a/gnocchi/manifests/params.pp b/gnocchi/manifests/params.pp index d97aa34aa..d170b9cb5 100644 --- a/gnocchi/manifests/params.pp +++ b/gnocchi/manifests/params.pp @@ -10,6 +10,8 @@ $api_service_name = 'openstack-gnocchi-api' $indexer_package_name = 'openstack-gnocchi-indexer-sqlalchemy' $carbonara_package_name = 'openstack-gnocchi-carbonara' + $statsd_package_name = 'openstack-gnocchi-statsd' + $statsd_service_name = 'openstack-gnocchi-statsd' $gnocchi_wsgi_script_path = '/var/www/cgi-bin/gnocchi' $gnocchi_wsgi_script_source = '/usr/lib/python2.7/site-packages/gnocchi/rest/app.wsgi' } @@ -20,6 +22,8 @@ $api_service_name = 'gnocchi-api' $indexer_package_name = 'gnocchi-indexer-sqlalchemy' $carbonara_package_name = 'gnocchi-carbonara' + $statsd_package_name = 'gnocchi-statsd' + $statsd_service_name = 'gnocchi-statsd' $gnocchi_wsgi_script_path = '/usr/lib/cgi-bin/gnocchi' $gnocchi_wsgi_script_source = '/usr/share/gnocchi-common/app.wsgi' } diff --git a/gnocchi/manifests/statsd.pp b/gnocchi/manifests/statsd.pp new file mode 100644 index 000000000..6f27ee64f --- /dev/null +++ b/gnocchi/manifests/statsd.pp @@ -0,0 +1,82 @@ +# Installs & configure the gnocchi statsd service +# +# == Parameters +# +# [*enabled*] +# (optional) Should the service be enabled. +# Defaults to true +# +# [*package_ensure*] +# (optional) ensure state for package. +# Defaults to 'present' +# +# [*manage_service*] +# (optional) Whether the service should be managed by Puppet. +# Defaults to true. +# +# [*resource_id*] +# (optional) Resource UUID to use to identify statsd in Gnocchi. +# Defaults to undef. +# +# [*user_id*] +# (optional) User UUID to use to identify statsd in Gnocchi. +# Defaults to undef. +# +# [*project_id*] +# (optional) Project UUID to use to identify statsd in Gnocchi. +# Defaults to undef. +# +# [*archive_policy_name*] +# (optional) Archive policy name to use when creating metrics. +# Defaults to undef. +# +# [*flush_delay*] +# (optional) Delay between flushes. +# Defaults to undef. +# +class gnocchi::statsd ( + $resource_id, + $user_id, + $project_id, + $flush_delay, + $archive_policy_name = undef, + $manage_service = true, + $enabled = true, + $package_ensure = 'present', +) inherits gnocchi::params { + + Gnocchi_config<||> ~> Service['gnocchi-statsd'] + Package['gnocchi-statsd'] -> Service['gnocchi-statsd'] + + package { 'gnocchi-statsd': + ensure => $package_ensure, + name => $::gnocchi::params::statsd_package_name, + tag => ['openstack', 'gnocchi-package'], + } + + if $manage_service { + if $enabled { + $service_ensure = 'running' + } else { + $service_ensure = 'stopped' + } + } + + service { 'gnocchi-statsd': + ensure => $service_ensure, + name => $::gnocchi::params::statsd_service_name, + enable => $enabled, + hasstatus => true, + hasrestart => true, + tag => ['gnocchi-service', 'gnocchi-db-sync-service'], + } + + gnocchi_config { + 'statsd/resource_id' : value => $resource_id; + 'statsd/user_id' : value => $user_id; + 'statsd/project_id' : value => $project_id; + 'statsd/archive_policy_name' : value => $archive_policy_name; + 'statsd/flush_delay' : value => $flush_delay; + } + +} diff --git a/gnocchi/metadata.json b/gnocchi/metadata.json index 739ea7044..69bfb42b0 100644 --- a/gnocchi/metadata.json +++ b/gnocchi/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-gnocchi", - "version": "5.0.0", + "version": "7.0.0", "author": "OpenStack Contributors", "summary": "Puppet module for OpenStack Gnocchi", "license": "Apache-2.0", @@ -32,8 +32,8 @@ ], "dependencies": [ { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" } + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" } ] } diff --git a/gnocchi/spec/acceptance/basic_gnocchi_spec.rb b/gnocchi/spec/acceptance/basic_gnocchi_spec.rb index 549b38971..29abe659a 100644 --- a/gnocchi/spec/acceptance/basic_gnocchi_spec.rb +++ b/gnocchi/spec/acceptance/basic_gnocchi_spec.rb @@ -40,6 +40,13 @@ class { '::gnocchi::storage::file': } class { '::gnocchi::wsgi::apache': ssl => false, } + class { '::gnocchi::statsd': + archive_policy_name => 'high', + flush_delay => '100', + resource_id => '07f26121-5777-48ba-8a0b-d70468133dd9', + user_id => 'f81e9b1f-9505-4298-bc33-43dfbd9a973b', + project_id => '203ef419-e73f-4b8a-a73f-3d599a72b18d', + } } } EOS diff --git a/gnocchi/spec/classes/gnocchi_statsd_spec.rb b/gnocchi/spec/classes/gnocchi_statsd_spec.rb new file mode 100644 index 000000000..14fcfbc1b --- /dev/null +++ b/gnocchi/spec/classes/gnocchi_statsd_spec.rb @@ -0,0 +1,103 @@ +require 'spec_helper' + +describe 'gnocchi::statsd' do + + let :params do + { :enabled => true, + :manage_service => true, + :resource_id => '07f26121-5777-48ba-8a0b-d70468133dd9', + :user_id => '07f26121-5777-48ba-8a0b-d70468133dd9', + :project_id => '07f26121-5777-48ba-8a0b-d70468133dd9', + :archive_policy_name => 'high', + :flush_delay => '200', + } + end + + shared_examples_for 'gnocchi-statsd' do + + it { is_expected.to contain_class('gnocchi::params') } + + it 'installs gnocchi-statsd package' do + is_expected.to contain_package('gnocchi-statsd').with( + :ensure => 'present', + :name => platform_params[:statsd_package_name], + :tag => ['openstack', 'gnocchi-package'], + ) + end + + it 'configures gnocchi statsd' do + is_expected.to contain_gnocchi_config('statsd/resource_id').with_value('07f26121-5777-48ba-8a0b-d70468133dd9') + is_expected.to contain_gnocchi_config('statsd/user_id').with_value('07f26121-5777-48ba-8a0b-d70468133dd9') + is_expected.to contain_gnocchi_config('statsd/project_id').with_value('07f26121-5777-48ba-8a0b-d70468133dd9') + is_expected.to contain_gnocchi_config('statsd/archive_policy_name').with_value('high') + is_expected.to contain_gnocchi_config('statsd/flush_delay').with_value('200') + end + + [{:enabled => true}, {:enabled => false}].each do |param_hash| + context "when service should be #{param_hash[:enabled] ? 'enabled' : 'disabled'}" do + before do + params.merge!(param_hash) + end + + it 'configures gnocchi-statsd service' do + is_expected.to contain_service('gnocchi-statsd').with( + :ensure => (params[:manage_service] && params[:enabled]) ? 'running' : 'stopped', + :name => platform_params[:statsd_service_name], + :enable => params[:enabled], + :hasstatus => true, + :hasrestart => true, + :tag => ['gnocchi-service', 'gnocchi-db-sync-service'], + ) + end + end + end + + context 'with disabled service managing' do + before do + params.merge!({ + :manage_service => false, + :enabled => false }) + end + + it 'configures gnocchi-statsd service' do + is_expected.to contain_service('gnocchi-statsd').with( + :ensure => nil, + :name => platform_params[:statsd_service_name], + :enable => false, + :hasstatus => true, + :hasrestart => true, + :tag => ['gnocchi-service', 'gnocchi-db-sync-service'], + ) + end + end + end + + context 'on Debian platforms' do + let :facts do + { :osfamily => 'Debian', + :operatingsystem => 'Debian' } + end + + let :platform_params do + { :statsd_package_name => 'gnocchi-statsd', + :statsd_service_name => 'gnocchi-statsd' } + end + + it_configures 'gnocchi-statsd' + end + + context 'on RedHat platforms' do + let :facts do + { :osfamily => 'RedHat', + :operatingsystem => 'RedHat' } + end + + let :platform_params do + { :statsd_package_name => 'openstack-gnocchi-statsd', + :statsd_service_name => 'openstack-gnocchi-statsd' } + end + + it_configures 'gnocchi-statsd' + end + +end diff --git a/gnocchi/spec/spec_helper_acceptance.rb b/gnocchi/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/gnocchi/spec/spec_helper_acceptance.rb +++ b/gnocchi/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/heat/CHANGELOG.md b/heat/CHANGELOG.md index 9b9a1b34f..5fc6ac95a 100644 --- a/heat/CHANGELOG.md +++ b/heat/CHANGELOG.md @@ -1,3 +1,46 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes +- change section name for AMQP qpid parameters +- change section name for AMQP rabbit parameters +- update rpc_backend default parameter +- cleanup configure_delegated_roles deprecated parameter + +####Features +- add support for RabbitMQ connection heartbeat +- keystone/auth: make service description configurable +- add tag to package and service resources +- add heat::config class +- expose RPC response timeout as a puppet parameter +- support setting instance_user to an empty string +- add heat::db::sync +- add an ability to manage use_stderr parameter +- reflect provider change in puppet-openstacklib +- put all the logging related parameters to the logging class +- add rabbit_ha_queues option +- improve heat::keystone::domain +- remove POSIX users, groups, and file modes +- use postgresql lib class for psycopg package +- move deps & external hooks into a standalone class +- introduce heat::db class +- make the role for heat_stack_user configurable +- allow to not manage Keystone domain +- add hooks for external install & svc management + +####Bugfixes +- rely on autorequire for config resource ordering +- fix up doc string for workers variable + +####Maintenance +- acceptance: enable debug & verbosity for OpenStack logs +- initial msync run for all Puppet OpenStack modules +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- acceptance: use common bits from puppet-openstack-integration + ##2015-10-14 - 6.1.0 ###Summary diff --git a/heat/README.md b/heat/README.md index 7c0486994..aecc697bb 100644 --- a/heat/README.md +++ b/heat/README.md @@ -1,7 +1,7 @@ puppet-heat ============= -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2 - Liberty #### Table of Contents diff --git a/heat/Rakefile b/heat/Rakefile index ed79bead4..9692ffdd2 100644 --- a/heat/Rakefile +++ b/heat/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/heat/manifests/init.pp b/heat/manifests/init.pp index d119f0f9a..87a33441b 100644 --- a/heat/manifests/init.pp +++ b/heat/manifests/init.pp @@ -404,6 +404,8 @@ if $rpc_backend == 'qpid' { + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + heat_config { 'oslo_messaging_qpid/qpid_hostname' : value => $qpid_hostname; 'oslo_messaging_qpid/qpid_port' : value => $qpid_port; diff --git a/heat/metadata.json b/heat/metadata.json index 5798063a1..0fba24bdc 100644 --- a/heat/metadata.json +++ b/heat/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-heat", - "version": "6.1.0", + "version": "7.0.0", "author": "eNovance and OpenStack Contributors", "summary": "Puppet module for OpenStack Heat", "license": "Apache-2.0", @@ -32,8 +32,8 @@ "description": "Installs and configures OpenStack Heat (Orchestration).", "dependencies": [ { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" } + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" } ] } diff --git a/heat/spec/spec_helper_acceptance.rb b/heat/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/heat/spec/spec_helper_acceptance.rb +++ b/heat/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/horizon/CHANGELOG.md b/horizon/CHANGELOG.md index e7f8ab0dc..8966ea145 100644 --- a/horizon/CHANGELOG.md +++ b/horizon/CHANGELOG.md @@ -1,3 +1,34 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes + +####Features +- allow Orchestration of Image Backend +- add new parameters for multidomain support +- add api_versions parameter +- remove hardcoded 'neutron_options' list +- remove hardcoded 'cinder_options' list +- remove hardcoded 'hypervisor_options' list +- add ssl_no_verify parameter +- make redirect type configurable +- add CUSTOM_THEME_PATH variable +- add tag to package resource +- add cache_backend and cache_options params + +####Bugfixes +- do not run collectstatic or compress in Debian +- remove duplicate key for ':servername' + +####Maintenance +- acceptance: enable debug & verbosity for OpenStack logs +- initial msync run for all Puppet OpenStack modules +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- fix rspec 3.x syntax + ##2015-10-15 - 6.1.0 ###Summary diff --git a/horizon/README.md b/horizon/README.md index 686b896be..e358aed32 100644 --- a/horizon/README.md +++ b/horizon/README.md @@ -1,7 +1,7 @@ horizon ======= -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2 - Liberty #### Table of Contents diff --git a/horizon/Rakefile b/horizon/Rakefile index ed79bead4..9692ffdd2 100644 --- a/horizon/Rakefile +++ b/horizon/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/horizon/manifests/init.pp b/horizon/manifests/init.pp index 5f3a5c946..fd404ea39 100644 --- a/horizon/manifests/init.pp +++ b/horizon/manifests/init.pp @@ -385,7 +385,7 @@ # who has since fixed their packaging. # See I813b5f6067bb6ecce279cab7278d9227c4d31d28 for the original history # behind this section. - if $::os_package_type == 'redhat' { + if $::os_package_type == 'rpm' { exec { 'refresh_horizon_django_cache': command => "${::horizon::params::manage_py} collectstatic --noinput --clear && ${::horizon::params::manage_py} compress --force", refreshonly => true, diff --git a/horizon/metadata.json b/horizon/metadata.json index 9e5bab6ea..8db0311d9 100644 --- a/horizon/metadata.json +++ b/horizon/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-horizon", - "version": "6.1.0", + "version": "7.0.0", "author": "Puppet Labs and OpenStack Contributors", "summary": "Puppet module for OpenStack Horizon", "license": "Apache-2.0", diff --git a/horizon/spec/classes/horizon_init_spec.rb b/horizon/spec/classes/horizon_init_spec.rb index 75ea55c32..f0adbef78 100644 --- a/horizon/spec/classes/horizon_init_spec.rb +++ b/horizon/spec/classes/horizon_init_spec.rb @@ -32,7 +32,7 @@ ) } it { - if facts[:os_package_type] == 'redhat' + if facts[:os_package_type] == 'rpm' is_expected.to contain_exec('refresh_horizon_django_cache').with({ :command => '/usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear && /usr/share/openstack-dashboard/manage.py compress --force', :refreshonly => true, @@ -42,7 +42,7 @@ end } it { - if facts[:os_package_type] == 'redhat' + if facts[:os_package_type] == 'rpm' is_expected.to contain_concat(platforms_params[:config_file]).that_notifies('Exec[refresh_horizon_django_cache]') else is_expected.to_not contain_concat(platforms_params[:config_file]).that_notifies('Exec[refresh_horizon_django_cache]') @@ -180,7 +180,7 @@ end it { - if facts[:os_package_type] == 'redhat' + if facts[:os_package_type] == 'rpm' is_expected.to contain_exec('refresh_horizon_django_cache') else is_expected.to_not contain_exec('refresh_horizon_django_cache') diff --git a/horizon/spec/spec_helper_acceptance.rb b/horizon/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/horizon/spec/spec_helper_acceptance.rb +++ b/horizon/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/ironic/CHANGELOG.md b/ironic/CHANGELOG.md index 3a27b6822..664df3db2 100644 --- a/ironic/CHANGELOG.md +++ b/ironic/CHANGELOG.md @@ -1,3 +1,53 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes +- change section name for AMQP qpid parameters +- change section name for AMQP rabbit parameters + +####Features +- add heat::db::sync +- add bifrost manifest +- reflect provider change in puppet-openstacklib +- put all the logging related parameters to the logging class +- add ironic-inspector support +- simplify rpc_backend parameter +- introduce ironic::db class +- db: Use postgresql lib class for psycopg package +- allow customization of db sync command line +- allow customization of force_power_state_during_sync +- add ironic::config class +- add tag to package and service resources +- add support for identity_uri + +####Bugfixes +- rely on autorequire for config resource ordering +- fixed the comment for port in ironic api.pp +- add BOOTIF=${mac} to the inspector iPXE template + +####Maintenance +- acceptance: enable debug & verbosity for OpenStack logs +- initial msync run for all Puppet OpenStack modules +- fix rspec 3.x syntax +- acceptance: install openstack-selinux on redhat plateforms +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- acceptance: use common bits from puppet-openstack-integration + +##2015-10-14 - 6.1.0 +###Summary + +This is a feature and maintenance release in the Kilo series. + +####Features +- Create Heat Domain with Keystone_domain resource + +####Maintenance +- Remove deprecated parameter stack_user_domain +- acceptance: checkout stable/kilo puppet modules + ##2015-10-15 - 6.1.0 ###Summary diff --git a/ironic/README.md b/ironic/README.md index 89afab980..2567e6db1 100644 --- a/ironic/README.md +++ b/ironic/README.md @@ -1,7 +1,7 @@ puppet-ironic ============= -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2 - Liberty #### Table of Contents diff --git a/ironic/Rakefile b/ironic/Rakefile index ed79bead4..9692ffdd2 100644 --- a/ironic/Rakefile +++ b/ironic/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/ironic/manifests/init.pp b/ironic/manifests/init.pp index 41a2d457f..aa5442c0a 100644 --- a/ironic/manifests/init.pp +++ b/ironic/manifests/init.pp @@ -369,6 +369,9 @@ } if $rpc_backend == 'ironic.openstack.common.rpc.impl_qpid' or $rpc_backend == 'qpid' { + + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + ironic_config { 'oslo_messaging_qpid/qpid_hostname': value => $qpid_hostname; 'oslo_messaging_qpid/qpid_port': value => $qpid_port; diff --git a/ironic/metadata.json b/ironic/metadata.json index e28b32457..b3fc4f31e 100644 --- a/ironic/metadata.json +++ b/ironic/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-ironic", - "version": "6.1.0", + "version": "7.0.0", "author": "eNovance and OpenStack Contributors", "summary": "Puppet module for OpenStack Ironic", "license": "Apache-2.0", @@ -32,9 +32,9 @@ "description": "Installs and configures OpenStack Ironic (Bare metal).", "dependencies": [ { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/vcsrepo", "version_requirement": ">=1.3.0 <2.0.0"} ] } diff --git a/ironic/spec/spec_helper_acceptance.rb b/ironic/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/ironic/spec/spec_helper_acceptance.rb +++ b/ironic/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/ironic/templates/inspector_ipxe.erb b/ironic/templates/inspector_ipxe.erb index 7362ca0bf..d5129f30d 100644 --- a/ironic/templates/inspector_ipxe.erb +++ b/ironic/templates/inspector_ipxe.erb @@ -2,6 +2,6 @@ dhcp -kernel http://<%= @dnsmasq_local_ip %>:8088/agent.kernel ipa-inspection-callback-url=http://<%= @dnsmasq_local_ip %>:5050/v1/continue systemd.journald.forward_to_console=yes +kernel http://<%= @dnsmasq_local_ip %>:8088/agent.kernel ipa-inspection-callback-url=http://<%= @dnsmasq_local_ip %>:5050/v1/continue systemd.journald.forward_to_console=yes BOOTIF=${mac} initrd http://<%= @dnsmasq_local_ip %>:8088/agent.ramdisk boot diff --git a/manila/CHANGELOG.md b/manila/CHANGELOG.md index 3f93342c0..5d7165874 100644 --- a/manila/CHANGELOG.md +++ b/manila/CHANGELOG.md @@ -1,3 +1,37 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes +- rabbitmq: do not manage rabbitmq service anymore +- move qpid settings to oslo_messaging_qpid section + +####Features +- keystone/auth: make service description configurable +- add related parameters to manila::quota +- add tag to package and service resources +- add support to configure GlusterFS drivers with Manila shares +- reflect provider change in puppet-openstacklib +- put all the logging related parameters to the logging class +- simplify rpc_backend parameter +- add options to enable Manila to run with NFS-Ganesha backend +- introduce manila::db class +- db: Use postgresql lib class for psycopg package +- add related parameters to oslo_messaging_amqp section + +####Bugfixes +- rely on autorequire for config resource ordering +- api: require ::keystone::python + +####Maintenance +- acceptance: enable debug & verbosity for OpenStack logs +- initial msync run for all Puppet OpenStack modules +- fix rspec 3.x syntax +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- acceptance: use common bits from puppet-openstack-integration + ##2015-10-10 - 6.1.0 ### Summary diff --git a/manila/README.md b/manila/README.md index 4cd3b6b68..403b3d96f 100644 --- a/manila/README.md +++ b/manila/README.md @@ -1,7 +1,7 @@ manila ======= -6.1.0 - 2015.1.0 - Kilo +7.0.0 - 2015.2 - Liberty #### Table of Contents diff --git a/manila/Rakefile b/manila/Rakefile index ed79bead4..9692ffdd2 100644 --- a/manila/Rakefile +++ b/manila/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/manila/manifests/init.pp b/manila/manifests/init.pp index 2232e4fde..b2b0b55f0 100644 --- a/manila/manifests/init.pp +++ b/manila/manifests/init.pp @@ -450,6 +450,8 @@ if $rpc_backend == 'manila.openstack.common.rpc.impl_qpid' or $rpc_backend == 'qpid' { + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + if ! $qpid_password { fail('Please specify a qpid_password parameter.') } diff --git a/manila/manifests/keystone/auth.pp b/manila/manifests/keystone/auth.pp index dcc4be1cf..677ebd6ef 100644 --- a/manila/manifests/keystone/auth.pp +++ b/manila/manifests/keystone/auth.pp @@ -41,6 +41,36 @@ # (optional) The endpoint's internal url. (Defaults to 'http://127.0.0.1:8786/v1/%(tenant_id)s') # This url should *not* contain any trailing '/'. # +# [*password_v2*] +# Password for Manila v2 user. Optional. Defaults to undef. +# +# [*email_v2*] +# Email for Manila v2 user. Optional. Defaults to 'manilav2@localhost'. +# +# [*auth_name_v2*] +# Username for Manila v2 service. Optional. Defaults to 'manilav2'. +# +# [*configure_endpoint_v2*] +# Should Manila v2 endpoint be configured? Optional. Defaults to 'true'. +# +# [*service_type_v2*] +# Type of service v2. Optional. Defaults to 'sharev2'. +# +# [*service_description_v2*] +# Description for keystone service v2. Optional. Defaults to 'Manila Service v2'. +# +# [*public_url_v2*] +# (optional) The v2 endpoint's public url. (Defaults to 'http://127.0.0.1:8786/v2/%(tenant_id)s') +# This url should *not* contain any trailing '/'. +# +# [*admin_url_v2*] +# (optional) The endpoint's admin url. (Defaults to 'http://127.0.0.1:8786/v2/%(tenant_id)s') +# This url should *not* contain any trailing '/'. +# +# [*internal_url_v2*] +# (optional) The endpoint's internal url. (Defaults to 'http://127.0.0.1:8786/v2/%(tenant_id)s') +# This url should *not* contain any trailing '/'. +# # [*share_version*] # (optional) DEPRECATED: Use public_url, internal_url and admin_url instead. # API version endpoint. (Defaults to 'v1') @@ -96,27 +126,44 @@ # class manila::keystone::auth ( $password, - $auth_name = 'manila', - $email = 'manila@localhost', - $tenant = 'services', - $configure_endpoint = true, - $service_type = 'share', - $service_description = 'Manila Service', - $region = 'RegionOne', - $public_url = 'http://127.0.0.1:8786/v1/%(tenant_id)s', - $admin_url = 'http://127.0.0.1:8786/v1/%(tenant_id)s', - $internal_url = 'http://127.0.0.1:8786/v1/%(tenant_id)s', + $password_v2 = undef, + $auth_name_v2 = 'manilav2', + $auth_name = 'manila', + $email = 'manila@localhost', + $email_v2 = 'manilav2@localhost', + $tenant = 'services', + $configure_endpoint = true, + $configure_endpoint_v2 = true, + $service_type = 'share', + $service_type_v2 = 'sharev2', + $service_description = 'Manila Service', + $service_description_v2 = 'Manila Service v2', + $region = 'RegionOne', + $public_url = 'http://127.0.0.1:8786/v1/%(tenant_id)s', + $public_url_v2 = 'http://127.0.0.1:8786/v2/%(tenant_id)s', + $admin_url = 'http://127.0.0.1:8786/v1/%(tenant_id)s', + $admin_url_v2 = 'http://127.0.0.1:8786/v2/%(tenant_id)s', + $internal_url = 'http://127.0.0.1:8786/v1/%(tenant_id)s', + $internal_url_v2 = 'http://127.0.0.1:8786/v2/%(tenant_id)s', # DEPRECATED PARAMETERS - $share_version = undef, - $port = undef, - $public_protocol = undef, - $public_address = undef, - $internal_protocol = undef, - $internal_address = undef, - $admin_protocol = undef, - $admin_address = undef, + $share_version = undef, + $port = undef, + $public_protocol = undef, + $public_address = undef, + $internal_protocol = undef, + $internal_address = undef, + $admin_protocol = undef, + $admin_address = undef, ) { + # for interface backward compatibility, we can't enforce to set a new parameter + # so we take 'password' parameter by default but allow to override it. + if ! $password_v2 { + $password_v2_real = $password + } else { + $password_v2_real = $password_v2 + } + if $share_version { warning('The share_version parameter is deprecated, use public_url, internal_url and admin_url instead.') } @@ -180,6 +227,7 @@ } Keystone_user_role["${auth_name}@${tenant}"] ~> Service <| name == 'manila-api' |> + Keystone_user_role["${auth_name_v2}@${tenant}"] ~> Service <| name == 'manila-api' |> keystone::resource::service_identity { $auth_name: configure_user => true, @@ -196,4 +244,18 @@ internal_url => $internal_url_real, } + keystone::resource::service_identity { $auth_name_v2: + configure_user => true, + configure_user_role => true, + configure_endpoint => $configure_endpoint_v2, + service_type => $service_type_v2, + service_description => $service_description_v2, + region => $region, + password => $password_v2_real, + email => $email_v2, + tenant => $tenant, + public_url => $public_url_v2, + admin_url => $admin_url_v2, + internal_url => $internal_url_v2, + } } diff --git a/manila/manifests/qpid.pp b/manila/manifests/qpid.pp index c3da33d80..bea80de19 100644 --- a/manila/manifests/qpid.pp +++ b/manila/manifests/qpid.pp @@ -33,6 +33,8 @@ $realm = 'OPENSTACK' ) { + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + # only configure manila after the queue is up Class['qpid::server'] -> Package<| title == 'manila' |> diff --git a/manila/metadata.json b/manila/metadata.json index 48a597472..38b340dda 100644 --- a/manila/metadata.json +++ b/manila/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-manila", - "version": "6.1.0", + "version": "7.0.0", "author": "NetApp and OpenStack Contributors", "summary": "Puppet module for OpenStack Manila", "license": "Apache-2.0", @@ -33,10 +33,10 @@ "dependencies": [ { "name": "dprince/qpid", "version_requirement": ">=1.0.0 <2.0.0" }, { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, - { "name": "openstack/glance", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, + { "name": "openstack/glance", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/rabbitmq", "version_requirement": ">=2.0.2 <6.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" } + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" } ] } diff --git a/manila/spec/acceptance/basic_manila_spec.rb b/manila/spec/acceptance/basic_manila_spec.rb index 905746dc1..519496b87 100644 --- a/manila/spec/acceptance/basic_manila_spec.rb +++ b/manila/spec/acceptance/basic_manila_spec.rb @@ -6,59 +6,11 @@ it 'should work with no errors' do pp= <<-EOS - Exec { logoutput => 'on_failure' } - - # Common resources - case $::osfamily { - 'Debian': { - include ::apt - class { '::openstack_extras::repo::debian::ubuntu': - release => 'liberty', - repo => 'proposed', - package_require => true, - } - $package_provider = 'apt' - } - 'RedHat': { - class { '::openstack_extras::repo::redhat::redhat': - manage_rdo => false, - repo_hash => { - 'openstack-common-testing' => { - 'baseurl' => 'http://cbs.centos.org/repos/cloud7-openstack-common-testing/x86_64/os/', - 'descr' => 'openstack-common-testing', - 'gpgcheck' => 'no', - }, - 'openstack-liberty-testing' => { - 'baseurl' => 'http://cbs.centos.org/repos/cloud7-openstack-liberty-testing/x86_64/os/', - 'descr' => 'openstack-liberty-testing', - 'gpgcheck' => 'no', - }, - 'openstack-liberty-trunk' => { - 'baseurl' => 'http://trunk.rdoproject.org/centos7-liberty/current-passed-ci/', - 'descr' => 'openstack-liberty-trunk', - 'gpgcheck' => 'no', - }, - }, - } - package { 'openstack-selinux': ensure => 'latest' } - $package_provider = 'yum' - } - default: { - fail("Unsupported osfamily (${::osfamily})") - } - } - - class { '::mysql::server': } - - class { '::rabbitmq': - delete_guest_user => true, - package_provider => $package_provider, - } - - rabbitmq_vhost { '/': - provider => 'rabbitmqctl', - require => Class['rabbitmq'], - } + include ::openstack_integration + include ::openstack_integration::repos + include ::openstack_integration::rabbitmq + include ::openstack_integration::mysql + include ::openstack_integration::keystone rabbitmq_user { 'manila': admin => true, @@ -75,26 +27,6 @@ class { '::rabbitmq': require => Class['rabbitmq'], } - # Keystone resources, needed by Manila to run - class { '::keystone::db::mysql': - password => 'keystone', - } - class { '::keystone': - verbose => true, - debug => true, - database_connection => 'mysql://keystone:keystone@127.0.0.1/keystone', - admin_token => 'admin_token', - enabled => true, - } - class { '::keystone::roles::admin': - email => 'test@example.tld', - password => 'a_big_secret', - } - class { '::keystone::endpoint': - public_url => "https://${::fqdn}:5000/", - admin_url => "https://${::fqdn}:35357/", - } - # Manila resources class { '::manila': sql_connection => 'mysql://manila:a_big_secret@127.0.0.1/manila?charset=utf8', @@ -108,7 +40,8 @@ class { '::manila::db::mysql': password => 'a_big_secret', } class { '::manila::keystone::auth': - password => 'a_big_secret', + password => 'a_big_secret', + password_v2 => 'a_big_secret', } class { '::manila::client': } class { '::manila::compute::nova': } diff --git a/manila/spec/classes/manila_keystone_auth_spec.rb b/manila/spec/classes/manila_keystone_auth_spec.rb index 7047b5d1d..27acfb6db 100644 --- a/manila/spec/classes/manila_keystone_auth_spec.rb +++ b/manila/spec/classes/manila_keystone_auth_spec.rb @@ -3,7 +3,8 @@ describe 'manila::keystone::auth' do let :params do - {:password => 'pw'} + {:password => 'pw', + :password_v2 => 'pw2'} end describe 'with only required params' do @@ -25,6 +26,21 @@ :description => 'Manila Service' ) + is_expected.to contain_keystone_user('manilav2').with( + :ensure => 'present', + :password => 'pw2', + :email => 'manilav2@localhost', + ) + is_expected.to contain_keystone_user_role('manilav2@services').with( + :ensure => 'present', + :roles => ['admin'] + ) + is_expected.to contain_keystone_service('manilav2').with( + :ensure => 'present', + :type => 'sharev2', + :description => 'Manila Service v2' + ) + end it { is_expected.to contain_keystone_endpoint('RegionOne/manila').with( :ensure => 'present', @@ -32,16 +48,25 @@ :admin_url => 'http://127.0.0.1:8786/v1/%(tenant_id)s', :internal_url => 'http://127.0.0.1:8786/v1/%(tenant_id)s' ) } + it { is_expected.to contain_keystone_endpoint('RegionOne/manilav2').with( + :ensure => 'present', + :public_url => 'http://127.0.0.1:8786/v2/%(tenant_id)s', + :admin_url => 'http://127.0.0.1:8786/v2/%(tenant_id)s', + :internal_url => 'http://127.0.0.1:8786/v2/%(tenant_id)s' + ) } end context 'when overriding endpoint parameters' do before do params.merge!( - :region => 'RegionThree', - :public_url => 'https://10.0.42.1:4242/v42/%(tenant_id)s', - :admin_url => 'https://10.0.42.2:4242/v42/%(tenant_id)s', - :internal_url => 'https://10.0.42.3:4242/v42/%(tenant_id)s' + :region => 'RegionThree', + :public_url => 'https://10.0.42.1:4242/v42/%(tenant_id)s', + :admin_url => 'https://10.0.42.2:4242/v42/%(tenant_id)s', + :internal_url => 'https://10.0.42.3:4242/v42/%(tenant_id)s', + :public_url_v2 => 'https://10.0.42.1:4242/v43/%(tenant_id)s', + :admin_url_v2 => 'https://10.0.42.2:4242/v43/%(tenant_id)s', + :internal_url_v2 => 'https://10.0.42.3:4242/v43/%(tenant_id)s' ) end @@ -51,6 +76,12 @@ :admin_url => 'https://10.0.42.2:4242/v42/%(tenant_id)s', :internal_url => 'https://10.0.42.3:4242/v42/%(tenant_id)s' )} + it { is_expected.to contain_keystone_endpoint('RegionThree/manilav2').with( + :ensure => 'present', + :public_url => 'https://10.0.42.1:4242/v43/%(tenant_id)s', + :admin_url => 'https://10.0.42.2:4242/v43/%(tenant_id)s', + :internal_url => 'https://10.0.42.3:4242/v43/%(tenant_id)s' + )} end context 'when deprecated endpoint parameters' do @@ -79,9 +110,11 @@ describe 'when endpoint should not be configured' do before do params.merge!( - :configure_endpoint => false + :configure_endpoint => false, + :configure_endpoint_v2 => false ) end it { is_expected.to_not contain_keystone_endpoint('RegionOne/manila') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/manilav2') } end end diff --git a/manila/spec/spec_helper_acceptance.rb b/manila/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/manila/spec/spec_helper_acceptance.rb +++ b/manila/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/neutron/CHANGELOG.md b/neutron/CHANGELOG.md index 600bae1c2..1f9add36f 100644 --- a/neutron/CHANGELOG.md +++ b/neutron/CHANGELOG.md @@ -1,3 +1,74 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes +- change section name for AMQP qpid parameters +- remove deprcated mysql options +- delete namespaces by default +- filename change for config-dir for ml2 cisco nexus mech driver +- unset deprecated parameter use_namespaces by default +- drop neutron_plugin_ovs type/provider + +####Features +- add driver option for metering agent +- split SR-IOV configuration file into driver and agent pieces +- disable third-party software management +- jumbo frames between instances +- remove POSIX users, groups, and file modes +- add support for RabbitMQ connection heartbeat +- keystone/auth: make service description configurable +- add tag to package and service resources +- allow neutron-dhcp-agent broadcast reply +- add neutron::db::sync and neutron::db +- add an ability to manage use_stderr parameter +- reflect provider change in puppet-openstacklib +- make vswitch optional for ovs agent configuration +- allow greater flexibility in the kind of parameter passed to ml2.pp +- deprecate nova_admin_* options +- drop nova_admin_tenant_id_setter +- db: Use postgresql lib class for psycopg package +- allow customization of db sync command line +- support for Nuage Neutron plugin +- suport for Big Switch ML2 plugin +- add an ability to set up drop_flows_on_start option +- use json output instead plain-text +- use json instead of regexp to parse subnet data +- simplify rpc_backend parameter +- add support for prevent_arp_spoofing option +- add support for rpc_response_timeout option +- remove use of template based config from cisco ml2 nexus plugin +- update Cisco ML2 configuration +- make package change run the db_sync +- accommodate for VPNaaS package name change in RDO + +####Bugfixes +- rely on autorequire for config resource ordering +- make sure Neutron_config is applied before Neutron_network +- make sure Nova_admin_tenant_id_setter is executed after Keystone_endpoint +- add workaround for v2 tenant names +- swap Facter[].value with Facter.value() +- dbsync should only notify services that use the db +- map allocation_pool property to set +- restart Neutron agents if packages are updated +- remove unecessary constraint between Package and Config resource +- fix 'shared' parameter check in neutron_network provider + +####Maintenance +- acceptance: enable debug & verbosity for OpenStack logs +- remove unused variable in params.pp +- initial msync run for all Puppet OpenStack modules +- lint: Add and fix missing parameter documentation +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- add test coverage for neutron::db::mysql +- fix rspec 3.x syntax +- acceptance: use common bits from puppet-openstack-integration +- fix unit tests against Puppet 4.3.0 +- acceptance: enable loadbalancer service_plugin +- acceptance: test OVS bridge listing + ##2015-10-10 - 6.1.0 ### Summary diff --git a/neutron/README.md b/neutron/README.md index 85cc5f0d8..c336cd772 100644 --- a/neutron/README.md +++ b/neutron/README.md @@ -1,7 +1,7 @@ neutron =================================== -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2.0 - Liberty #### Table of Contents diff --git a/neutron/Rakefile b/neutron/Rakefile index ed79bead4..9692ffdd2 100644 --- a/neutron/Rakefile +++ b/neutron/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/neutron/manifests/agents/ml2/ovs.pp b/neutron/manifests/agents/ml2/ovs.pp index 5540f3b32..e4291733f 100644 --- a/neutron/manifests/agents/ml2/ovs.pp +++ b/neutron/manifests/agents/ml2/ovs.pp @@ -109,10 +109,15 @@ # (optional) Enable or not ARP Spoofing Protection # Defaults to true # +# [*extensions*] +# (optional) Extensions list to use +# Defaults to [] +# class neutron::agents::ml2::ovs ( $package_ensure = 'present', $enabled = true, $manage_service = true, + $extensions = [], $bridge_uplinks = [], $bridge_mappings = [], $integration_bridge = 'br-int', @@ -182,6 +187,7 @@ 'agent/enable_distributed_routing': value => $enable_distributed_routing; 'agent/drop_flows_on_start': value => $drop_flows_on_start; 'agent/prevent_arp_spoofing': value => $prevent_arp_spoofing; + 'agent/extensions': value => join(any2array($extensions), ','); 'ovs/integration_bridge': value => $integration_bridge; } diff --git a/neutron/manifests/agents/ml2/sriov.pp b/neutron/manifests/agents/ml2/sriov.pp index 7c5a24868..7ca05f0bd 100644 --- a/neutron/manifests/agents/ml2/sriov.pp +++ b/neutron/manifests/agents/ml2/sriov.pp @@ -52,7 +52,10 @@ # semicolon separated list of virtual functions to exclude from network_device. # The network_device in the mapping should appear in the physical_device_mappings list. # - +# [*extensions*] +# (optional) Extensions list to use +# Defaults to [] +# class neutron::agents::ml2::sriov ( $package_ensure = 'present', $enabled = true, @@ -60,6 +63,7 @@ $physical_device_mappings = [], $polling_interval = 2, $exclude_devices = [], + $extensions = [], ) { include ::neutron::params @@ -70,6 +74,7 @@ 'sriov_nic/polling_interval': value => $polling_interval; 'sriov_nic/exclude_devices': value => join($exclude_devices, ','); 'sriov_nic/physical_device_mappings': value => join($physical_device_mappings, ','); + 'agent/extensions': value => join(any2array($extensions), ','); } Package['neutron-sriov-nic-agent'] -> Neutron_sriov_agent_config <||> diff --git a/neutron/manifests/init.pp b/neutron/manifests/init.pp index 5a6e7c151..5e32bd003 100644 --- a/neutron/manifests/init.pp +++ b/neutron/manifests/init.pp @@ -43,7 +43,7 @@ # [*service_plugins*] # (optional) Advanced service modules. # Could be an array that can have these elements: -# router, firewall, lbaas, vpnaas, metering +# router, firewall, lbaas, vpnaas, metering, qos # Defaults to empty # # [*auth_strategy*] @@ -499,6 +499,9 @@ } if $rpc_backend == 'qpid' or $rpc_backend == 'neutron.openstack.common.rpc.impl_qpid' { + + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + neutron_config { 'oslo_messaging_qpid/qpid_hostname': value => $qpid_hostname; 'oslo_messaging_qpid/qpid_port': value => $qpid_port; diff --git a/neutron/manifests/plugins/ml2.pp b/neutron/manifests/plugins/ml2.pp index 2ddff7799..1a055be4c 100644 --- a/neutron/manifests/plugins/ml2.pp +++ b/neutron/manifests/plugins/ml2.pp @@ -29,6 +29,11 @@ # local, flat, vlan, gre, vxlan # Defaults to ['local', 'flat', 'vlan', 'gre', 'vxlan']. # +# [*extension_drivers*] +# (optional) Ordered list of extension driver entrypoints to be loaded +# from the neutron.ml2.extension_drivers namespace. +# Defaults to []. +# # [*tenant_network_types*] # (optional) Ordered list of network_types to allocate as tenant networks. # The value 'local' is only useful for single-box testing @@ -118,6 +123,7 @@ class neutron::plugins::ml2 ( $type_drivers = ['local', 'flat', 'vlan', 'gre', 'vxlan'], + $extension_drivers = [], $tenant_network_types = ['local', 'flat', 'vlan', 'gre', 'vxlan'], $mechanism_drivers = ['openvswitch', 'linuxbridge'], $flat_networks = '*', @@ -198,6 +204,7 @@ 'ml2/tenant_network_types': value => join(any2array($tenant_network_types), ','); 'ml2/mechanism_drivers': value => join(any2array($mechanism_drivers), ','); 'ml2/path_mtu': value => $path_mtu; + 'ml2/extension_drivers': value => join(any2array($extension_drivers), ','); 'securitygroup/enable_security_group': value => $enable_security_group; } diff --git a/neutron/manifests/server.pp b/neutron/manifests/server.pp index ad13f8e4d..f44ffd758 100644 --- a/neutron/manifests/server.pp +++ b/neutron/manifests/server.pp @@ -196,6 +196,10 @@ # (optional) Deprecated, does nothing. # Defaults to 'undef'. # +# [*qos_notification_drivers*] +# (optional) Drivers list to use to send the update notification +# Defaults to []. +# class neutron::server ( $package_ensure = 'present', $enabled = true, @@ -226,6 +230,7 @@ $max_l3_agents_per_router = 3, $min_l3_agents_per_router = 2, $l3_ha_net_cidr = '169.254.192.0/18', + $qos_notification_drivers = [], # DEPRECATED PARAMETERS $auth_host = 'localhost', $auth_port = '35357', @@ -301,6 +306,8 @@ } } + neutron_config { 'qos/notification_drivers': value => join(any2array($qos_notification_drivers), ',') } + if ($::neutron::params::server_package) { Package['neutron-server'] -> Neutron_api_config<||> Package['neutron-server'] -> Neutron_config<||> diff --git a/neutron/metadata.json b/neutron/metadata.json index 55bf1b074..ffac354ad 100644 --- a/neutron/metadata.json +++ b/neutron/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-neutron", - "version": "6.1.0", + "version": "7.0.0", "author": "OpenStack Contributors", "summary": "Puppet module for OpenStack Neutron", "license": "Apache-2.0", @@ -32,11 +32,11 @@ "description": "Installs and configures OpenStack Neutron (Networking).", "dependencies": [ { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, - { "name": "openstack/nova", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, + { "name": "openstack/nova", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, - { "name": "openstack/vswitch", "version_requirement": ">=2.0.0 <3.0.0" }, + { "name": "openstack/vswitch", "version_requirement": ">=3.0.0 <4.0.0" }, { "name": "duritong/sysctl", "version_requirement": ">=0.0.1 <1.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" } + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" } ] } diff --git a/neutron/spec/classes/neutron_agents_dhcp_spec.rb b/neutron/spec/classes/neutron_agents_dhcp_spec.rb index 3c38c4615..a06f47295 100644 --- a/neutron/spec/classes/neutron_agents_dhcp_spec.rb +++ b/neutron/spec/classes/neutron_agents_dhcp_spec.rb @@ -180,7 +180,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_agents_l3_spec.rb b/neutron/spec/classes/neutron_agents_l3_spec.rb index d939deec0..1a69ff4c4 100644 --- a/neutron/spec/classes/neutron_agents_l3_spec.rb +++ b/neutron/spec/classes/neutron_agents_l3_spec.rb @@ -158,7 +158,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_agents_lbaas_spec.rb b/neutron/spec/classes/neutron_agents_lbaas_spec.rb index e7ec0c630..8f3e2784c 100644 --- a/neutron/spec/classes/neutron_agents_lbaas_spec.rb +++ b/neutron/spec/classes/neutron_agents_lbaas_spec.rb @@ -134,8 +134,9 @@ class { 'neutron': rabbit_password => 'passw0rd' }" context 'on RedHat platforms' do let :facts do default_facts.merge( - { :osfamily => 'RedHat', - :concat_basedir => '/dne' + { :osfamily => 'RedHat', + :operatingsystemrelease => '7', + :concat_basedir => '/dne' } ) end diff --git a/neutron/spec/classes/neutron_agents_metadata_spec.rb b/neutron/spec/classes/neutron_agents_metadata_spec.rb index 9ddb044ed..c73cf7519 100644 --- a/neutron/spec/classes/neutron_agents_metadata_spec.rb +++ b/neutron/spec/classes/neutron_agents_metadata_spec.rb @@ -119,9 +119,10 @@ context 'on Red Hat platforms' do let :facts do - default_facts.merge( - { :osfamily => 'RedHat' } - ) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_agents_metering_spec.rb b/neutron/spec/classes/neutron_agents_metering_spec.rb index da2fbace7..78c67b855 100644 --- a/neutron/spec/classes/neutron_agents_metering_spec.rb +++ b/neutron/spec/classes/neutron_agents_metering_spec.rb @@ -133,7 +133,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_agents_ml2_linuxbridge_spec.rb b/neutron/spec/classes/neutron_agents_ml2_linuxbridge_spec.rb index 6f36adfa5..96e8d7105 100644 --- a/neutron/spec/classes/neutron_agents_ml2_linuxbridge_spec.rb +++ b/neutron/spec/classes/neutron_agents_ml2_linuxbridge_spec.rb @@ -162,7 +162,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_agents_ml2_ovs_spec.rb b/neutron/spec/classes/neutron_agents_ml2_ovs_spec.rb index c1591eaed..f4225273b 100644 --- a/neutron/spec/classes/neutron_agents_ml2_ovs_spec.rb +++ b/neutron/spec/classes/neutron_agents_ml2_ovs_spec.rb @@ -48,6 +48,7 @@ is_expected.to contain_neutron_agent_ovs('agent/arp_responder').with_value(p[:arp_responder]) is_expected.to contain_neutron_agent_ovs('agent/prevent_arp_spoofing').with_value(p[:prevent_arp_spoofing]) is_expected.to contain_neutron_agent_ovs('agent/drop_flows_on_start').with_value(p[:drop_flows_on_start]) + is_expected.to contain_neutron_agent_ovs('agent/extensions').with_value(['']) is_expected.to contain_neutron_agent_ovs('ovs/integration_bridge').with_value(p[:integration_bridge]) is_expected.to contain_neutron_agent_ovs('securitygroup/firewall_driver').\ with_value(p[:firewall_driver]) @@ -176,6 +177,16 @@ end end + context 'when supplying extensions for ML2 plugin' do + before :each do + params.merge!(:extensions => ['qos']) + end + + it 'configures extensions' do + is_expected.to contain_neutron_agent_ovs('agent/extensions').with_value(params[:extensions].join(',')) + end + end + context 'when enabling tunneling' do context 'without local ip address' do before :each do @@ -250,7 +261,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_agents_ml2_sriov_spec.rb b/neutron/spec/classes/neutron_agents_ml2_sriov_spec.rb index 4711a09d2..3ccae81c6 100644 --- a/neutron/spec/classes/neutron_agents_ml2_sriov_spec.rb +++ b/neutron/spec/classes/neutron_agents_ml2_sriov_spec.rb @@ -38,6 +38,7 @@ is_expected.to contain_neutron_sriov_agent_config('sriov_nic/polling_interval').with_value(p[:polling_interval]) is_expected.to contain_neutron_sriov_agent_config('sriov_nic/exclude_devices').with_value(p[:exclude_devices].join(',')) is_expected.to contain_neutron_sriov_agent_config('sriov_nic/physical_device_mappings').with_value(p[:physical_device_mappings].join(',')) + is_expected.to contain_neutron_sriov_agent_config('agent/extensions').with_value(['']) end @@ -82,6 +83,16 @@ is_expected.to contain_neutron_sriov_agent_config('sriov_nic/physical_device_mappings').with_value(['physnet1:eth1']) end end + + context 'when supplying extensions for ML2 SR-IOV agent' do + before :each do + params.merge!(:extensions => ['qos']) + end + + it 'configures extensions' do + is_expected.to contain_neutron_sriov_agent_config('agent/extensions').with_value(params[:extensions].join(',')) + end + end end context 'on Debian platforms' do @@ -99,7 +110,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_client_spec.rb b/neutron/spec/classes/neutron_client_spec.rb index 80de68b3d..0dc005722 100644 --- a/neutron/spec/classes/neutron_client_spec.rb +++ b/neutron/spec/classes/neutron_client_spec.rb @@ -18,7 +18,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it { is_expected.to contain_class('neutron::client') } diff --git a/neutron/spec/classes/neutron_db_spec.rb b/neutron/spec/classes/neutron_db_spec.rb index 1672b6e81..6c1b1a02f 100644 --- a/neutron/spec/classes/neutron_db_spec.rb +++ b/neutron/spec/classes/neutron_db_spec.rb @@ -65,7 +65,7 @@ context 'on Redhat platforms' do let :facts do - { :osfamily => 'RedHat', + { :osfamily => 'RedHat', :operatingsystemrelease => '7.1', } end diff --git a/neutron/spec/classes/neutron_init_spec.rb b/neutron/spec/classes/neutron_init_spec.rb index d51f47346..f75266bca 100644 --- a/neutron/spec/classes/neutron_init_spec.rb +++ b/neutron/spec/classes/neutron_init_spec.rb @@ -449,12 +449,12 @@ shared_examples_for 'with service_plugins' do before do params.merge!( - :service_plugins => ['router','firewall','lbaas','vpnaas','metering'] + :service_plugins => ['router','firewall','lbaas','vpnaas','metering','qos'] ) end it do - is_expected.to contain_neutron_config('DEFAULT/service_plugins').with_value('router,firewall,lbaas,vpnaas,metering') + is_expected.to contain_neutron_config('DEFAULT/service_plugins').with_value('router,firewall,lbaas,vpnaas,metering,qos') end end @@ -514,7 +514,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_plugins_cisco_spec.rb b/neutron/spec/classes/neutron_plugins_cisco_spec.rb index 2ac2f1666..985d51c9b 100644 --- a/neutron/spec/classes/neutron_plugins_cisco_spec.rb +++ b/neutron/spec/classes/neutron_plugins_cisco_spec.rb @@ -168,7 +168,10 @@ class { 'neutron': rabbit_password => 'passw0rd' }" context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it_configures 'default cisco plugin' diff --git a/neutron/spec/classes/neutron_plugins_midonet_spec.rb b/neutron/spec/classes/neutron_plugins_midonet_spec.rb index d0cd08b2f..40d9bf5cc 100644 --- a/neutron/spec/classes/neutron_plugins_midonet_spec.rb +++ b/neutron/spec/classes/neutron_plugins_midonet_spec.rb @@ -69,7 +69,10 @@ class { 'neutron': rabbit_password => 'passw0rd' } context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat'}) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it_configures 'neutron midonet plugin' end diff --git a/neutron/spec/classes/neutron_plugins_ml2_bigswitch_restproxy_spec.rb b/neutron/spec/classes/neutron_plugins_ml2_bigswitch_restproxy_spec.rb index 42fc79e78..a21564086 100644 --- a/neutron/spec/classes/neutron_plugins_ml2_bigswitch_restproxy_spec.rb +++ b/neutron/spec/classes/neutron_plugins_ml2_bigswitch_restproxy_spec.rb @@ -70,7 +70,10 @@ class { 'neutron': context 'on RedHat platforms' do let :facts do - default_facts.merge({:osfamily => 'RedHat'}) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it_configures 'neutron bigswitch ml2 restproxy' diff --git a/neutron/spec/classes/neutron_plugins_ml2_bigswitch_spec.rb b/neutron/spec/classes/neutron_plugins_ml2_bigswitch_spec.rb index 80ace6c08..19a506d98 100644 --- a/neutron/spec/classes/neutron_plugins_ml2_bigswitch_spec.rb +++ b/neutron/spec/classes/neutron_plugins_ml2_bigswitch_spec.rb @@ -44,7 +44,10 @@ class { 'neutron': context 'on RedHat platforms' do let :facts do - default_facts.merge({:osfamily => 'RedHat'}) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it_configures 'neutron plugin bigswitch ml2' diff --git a/neutron/spec/classes/neutron_plugins_ml2_cisco_nexus_spec.rb b/neutron/spec/classes/neutron_plugins_ml2_cisco_nexus_spec.rb index bacdd5f21..0fce4940b 100644 --- a/neutron/spec/classes/neutron_plugins_ml2_cisco_nexus_spec.rb +++ b/neutron/spec/classes/neutron_plugins_ml2_cisco_nexus_spec.rb @@ -89,7 +89,10 @@ class { 'neutron': begin context 'on RedHat platforms' do let :facts do - default_facts.merge({:osfamily => 'RedHat'}) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_plugins_ml2_cisco_spec.rb b/neutron/spec/classes/neutron_plugins_ml2_cisco_spec.rb index d6cdfd78c..3fa810b33 100644 --- a/neutron/spec/classes/neutron_plugins_ml2_cisco_spec.rb +++ b/neutron/spec/classes/neutron_plugins_ml2_cisco_spec.rb @@ -46,7 +46,10 @@ class { 'neutron': begin context 'on RedHat platforms' do let :facts do - default_facts.merge({:osfamily => 'RedHat'}) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it_configures 'neutron plugin cisco ml2' diff --git a/neutron/spec/classes/neutron_plugins_ml2_cisco_type_nexus_vxlan_spec.rb b/neutron/spec/classes/neutron_plugins_ml2_cisco_type_nexus_vxlan_spec.rb index affac4ba1..392de5980 100644 --- a/neutron/spec/classes/neutron_plugins_ml2_cisco_type_nexus_vxlan_spec.rb +++ b/neutron/spec/classes/neutron_plugins_ml2_cisco_type_nexus_vxlan_spec.rb @@ -49,7 +49,10 @@ class { 'neutron': begin context 'on RedHat platforms' do let :facts do - default_facts.merge({:osfamily => 'RedHat'}) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it_configures 'neutron cisco ml2 type nexus vxlan plugin' diff --git a/neutron/spec/classes/neutron_plugins_ml2_cisco_ucsm_spec.rb b/neutron/spec/classes/neutron_plugins_ml2_cisco_ucsm_spec.rb index 26c211072..a97df0f30 100644 --- a/neutron/spec/classes/neutron_plugins_ml2_cisco_ucsm_spec.rb +++ b/neutron/spec/classes/neutron_plugins_ml2_cisco_ucsm_spec.rb @@ -55,7 +55,10 @@ class { 'neutron': begin context 'on RedHat platforms' do let :facts do - default_facts.merge({:osfamily => 'RedHat'}) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it_configures 'neutron cisco ml2 ucsm plugin' diff --git a/neutron/spec/classes/neutron_plugins_ml2_spec.rb b/neutron/spec/classes/neutron_plugins_ml2_spec.rb index 9cdd08c23..dc52cb573 100644 --- a/neutron/spec/classes/neutron_plugins_ml2_spec.rb +++ b/neutron/spec/classes/neutron_plugins_ml2_spec.rb @@ -67,6 +67,7 @@ is_expected.to contain_neutron_plugin_ml2('ml2/type_drivers').with_value(p[:type_drivers].join(',')) is_expected.to contain_neutron_plugin_ml2('ml2/tenant_network_types').with_value(p[:tenant_network_types].join(',')) is_expected.to contain_neutron_plugin_ml2('ml2/mechanism_drivers').with_value(p[:mechanism_drivers].join(',')) + is_expected.to contain_neutron_plugin_ml2('ml2/extension_drivers').with_value(['']) is_expected.to contain_neutron_plugin_ml2('ml2/path_mtu').with_value(p[:path_mtu]) is_expected.to contain_neutron_plugin_ml2('ml2/physical_network_mtus').with_ensure('absent') end @@ -88,6 +89,16 @@ end end + context 'when using extension drivers for ML2 plugin' do + before :each do + params.merge!(:extension_drivers => ['port_security','qos']) + end + + it 'configures extension drivers' do + is_expected.to contain_neutron_plugin_ml2('ml2/extension_drivers').with_value(p[:extension_drivers].join(',')) + end + end + context 'configure ml2 with bad driver value' do before :each do params.merge!(:type_drivers => ['foobar']) @@ -266,7 +277,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_plugins_nuage_spec.rb b/neutron/spec/classes/neutron_plugins_nuage_spec.rb index 223422333..45114718c 100644 --- a/neutron/spec/classes/neutron_plugins_nuage_spec.rb +++ b/neutron/spec/classes/neutron_plugins_nuage_spec.rb @@ -62,7 +62,10 @@ class { 'neutron::server': auth_password => 'password' }" context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat'}) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it_configures 'neutron plugin nuage' @@ -88,7 +91,10 @@ class { 'neutron::server': auth_password => 'password' }" context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat'}) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it 'should create plugin symbolic link' do diff --git a/neutron/spec/classes/neutron_plugins_nvp_spec.rb b/neutron/spec/classes/neutron_plugins_nvp_spec.rb index 48a15d3ca..4501722ce 100644 --- a/neutron/spec/classes/neutron_plugins_nvp_spec.rb +++ b/neutron/spec/classes/neutron_plugins_nvp_spec.rb @@ -105,7 +105,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({:osfamily => 'RedHat'}) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_plugins_opencontrail_spec.rb b/neutron/spec/classes/neutron_plugins_opencontrail_spec.rb index 975e89b21..f404b4001 100644 --- a/neutron/spec/classes/neutron_plugins_opencontrail_spec.rb +++ b/neutron/spec/classes/neutron_plugins_opencontrail_spec.rb @@ -73,7 +73,10 @@ class { 'neutron': rabbit_password => 'passw0rd' }" context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :params do diff --git a/neutron/spec/classes/neutron_plugins_plumgrid_spec.rb b/neutron/spec/classes/neutron_plugins_plumgrid_spec.rb index 398568d49..0788f0a37 100644 --- a/neutron/spec/classes/neutron_plugins_plumgrid_spec.rb +++ b/neutron/spec/classes/neutron_plugins_plumgrid_spec.rb @@ -92,7 +92,10 @@ class { 'neutron': rabbit_password => 'passw0rd' }" context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat'}) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it 'should create plugin symbolic link' do diff --git a/neutron/spec/classes/neutron_policy_spec.rb b/neutron/spec/classes/neutron_policy_spec.rb index df9016565..8b9a4866c 100644 --- a/neutron/spec/classes/neutron_policy_spec.rb +++ b/neutron/spec/classes/neutron_policy_spec.rb @@ -39,7 +39,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it_configures 'neutron policies' diff --git a/neutron/spec/classes/neutron_server_notifications_spec.rb b/neutron/spec/classes/neutron_server_notifications_spec.rb index 93ed7a23a..8045b500b 100644 --- a/neutron/spec/classes/neutron_server_notifications_spec.rb +++ b/neutron/spec/classes/neutron_server_notifications_spec.rb @@ -193,7 +193,10 @@ context 'on RedHat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_server_spec.rb b/neutron/spec/classes/neutron_server_spec.rb index acb40efdb..ba30697b0 100644 --- a/neutron/spec/classes/neutron_server_spec.rb +++ b/neutron/spec/classes/neutron_server_spec.rb @@ -34,7 +34,7 @@ :max_l3_agents_per_router => 3, :min_l3_agents_per_router => 2, :l3_ha_net_cidr => '169.254.192.0/18', - :allow_automatic_l3agent_failover => false + :allow_automatic_l3agent_failover => false, } end @@ -96,6 +96,7 @@ is_expected.to contain_neutron_config('DEFAULT/rpc_workers').with_value(facts[:processorcount]) is_expected.to contain_neutron_config('DEFAULT/agent_down_time').with_value(p[:agent_down_time]) is_expected.to contain_neutron_config('DEFAULT/router_scheduler_driver').with_value(p[:router_scheduler_driver]) + is_expected.to contain_neutron_config('qos/notification_drivers').with_value(['']) end context 'with manage_service as false' do @@ -182,6 +183,15 @@ is_expected.to contain_neutron_config('DEFAULT/allow_automatic_l3agent_failover').with_value(p[:allow_automatic_l3agent_failover]) end end + + context 'with qos_notification_drivers parameter' do + before :each do + params.merge!(:qos_notification_drivers => 'message_queue') + end + it 'should configure qos_notification_drivers' do + is_expected.to contain_neutron_config('qos/notification_drivers').with_value('message_queue') + end + end end shared_examples_for 'a neutron server with auth_admin_prefix set' do @@ -234,7 +244,10 @@ describe "with custom keystone auth_uri" do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end before do params.merge!({ @@ -253,7 +266,10 @@ describe "with custom keystone identity_uri" do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end before do params.merge!({ @@ -272,7 +288,10 @@ describe "with custom keystone identity_uri and auth_uri" do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end before do params.merge!({ @@ -292,7 +311,10 @@ describe "with custom auth region" do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end before do params.merge!({ @@ -326,8 +348,10 @@ context 'on RedHat platforms' do let :facts do default_facts.merge( - { :osfamily => 'RedHat', - :processorcount => '2' }) + { :osfamily => 'RedHat', + :operatingsystemrelease => '7', + :processorcount => '2' + }) end let :platform_params do diff --git a/neutron/spec/classes/neutron_services_fwaas_spec.rb b/neutron/spec/classes/neutron_services_fwaas_spec.rb index 39fc01695..ae13d5330 100644 --- a/neutron/spec/classes/neutron_services_fwaas_spec.rb +++ b/neutron/spec/classes/neutron_services_fwaas_spec.rb @@ -109,7 +109,10 @@ context 'on Red Hat platforms' do let :facts do - default_facts.merge({ :osfamily => 'RedHat' }) + default_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) end it_configures 'neutron fwaas service plugin' diff --git a/neutron/spec/spec_helper_acceptance.rb b/neutron/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/neutron/spec/spec_helper_acceptance.rb +++ b/neutron/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/nova/CHANGELOG.md b/nova/CHANGELOG.md index f415b3360..70592cd14 100644 --- a/nova/CHANGELOG.md +++ b/nova/CHANGELOG.md @@ -1,3 +1,63 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes +- change section name for AMQP qpid parameters +- change section name for AMQP rabbit parameters +- remove deprecated mysql_module +- do not manage python-greenlet anymore +- rabbitmq: do not manage rabbitmq service anymore +- remove openstackocci installation feature +- enable nova service by default + +####Features +- add tag to package and service resources +- add nova::db::sync +- add an ability to manage use_stderr parameter +- reflect provider change in puppet-openstacklib +- add nameservers (dns) parameters +- move os_region_name config option +- use auth_nova method to create nova network +- api: add default_floating_pool parameter +- db: Use postgresql lib class for psycopg package +- add support for RabbitMQ connection heartbeat +- move cinder_catalog_info to init +- don't add non-existent hosts to host aggregates +- make libvirt migration security configurable +- add region support to nova providers +- floating IP range support in Nova network +- rename neuton/url_timeout to neutron/timeout +- add upgrade_levels configuration options +- switch nova to leverage os_package_type fact +- use os_package_type for libvirt service name +- making instance_name_template configurable +- remove POSIX users, groups, and file modes +- allows the modification of the nova-api ports +- put all the logging related parameters to the logging class +- add kombu_reconnect_delay option +- update nova::db class to match other module pattern +- volume: allow to change catalog_info +- add config_drive_format option to nova_compute +- handle libvirt/cpu_model option +- add ability to set default baremetal filters + +####Bugfixes +- rely on autorequire for config resource ordering +- avoid empty notification driver +- fixed issue with rabbit_hosts parameter +- docfix: update default image_service param in doc + +####Maintenance +- fix rspec 3.x syntax +- acceptance: enable debug & verbosity for OpenStack logs +- initial msync run for all Puppet OpenStack modules +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- acceptance: use common bits from puppet-openstack-integration +- spec: enable webmock connect to IPv4 link-local + ##2015-10-10 - 6.1.0 ###Summary diff --git a/nova/README.md b/nova/README.md index 6ebbc18d3..521230f3c 100644 --- a/nova/README.md +++ b/nova/README.md @@ -1,7 +1,7 @@ nova ==== -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2.0 - Liberty #### Table of Contents diff --git a/nova/Rakefile b/nova/Rakefile index ed79bead4..9692ffdd2 100644 --- a/nova/Rakefile +++ b/nova/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/nova/manifests/init.pp b/nova/manifests/init.pp index 7afb0b036..d24a6404f 100644 --- a/nova/manifests/init.pp +++ b/nova/manifests/init.pp @@ -608,6 +608,9 @@ # we keep "nova.openstack.common.rpc.impl_qpid" for backward compatibility # but since Icehouse, "qpid" is enough. if $rpc_backend == 'nova.openstack.common.rpc.impl_qpid' or $rpc_backend == 'qpid' { + + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + nova_config { 'oslo_messaging_qpid/qpid_hostname': value => $qpid_hostname; 'oslo_messaging_qpid/qpid_port': value => $qpid_port; diff --git a/nova/manifests/qpid.pp b/nova/manifests/qpid.pp index 4aaae57bb..7bcac7eaf 100644 --- a/nova/manifests/qpid.pp +++ b/nova/manifests/qpid.pp @@ -32,6 +32,8 @@ $realm = 'OPENSTACK' ) { + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + # only configure nova after the queue is up Class['qpid::server'] -> Package<| title == 'nova-common' |> diff --git a/nova/metadata.json b/nova/metadata.json index 0850cc0cd..97e9f4d81 100644 --- a/nova/metadata.json +++ b/nova/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-nova", - "version": "6.1.0", + "version": "7.0.0", "author": "Puppet Labs and OpenStack Contributors", "summary": "Puppet module for OpenStack Nova", "license": "Apache-2.0", @@ -33,12 +33,12 @@ "dependencies": [ { "name": "dprince/qpid", "version_requirement": ">=1.0.0 <2.0.0" }, { "name": "duritong/sysctl", "version_requirement": ">=0.0.1 <1.0.0" }, - { "name": "openstack/cinder", "version_requirement": ">=6.0.0 <7.0.0" }, - { "name": "openstack/glance", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/cinder", "version_requirement": ">=7.0.0 <8.0.0" }, + { "name": "openstack/glance", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/rabbitmq", "version_requirement": ">=2.0.2 <6.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" } + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" } ] } diff --git a/nova/spec/spec_helper_acceptance.rb b/nova/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/nova/spec/spec_helper_acceptance.rb +++ b/nova/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/openstack_extras/CHANGELOG.md b/openstack_extras/CHANGELOG.md index e9bb37bb0..6c0a4b710 100644 --- a/openstack_extras/CHANGELOG.md +++ b/openstack_extras/CHANGELOG.md @@ -1,3 +1,22 @@ +##2015-11-24 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes +- repo: bump to Liberty by default + +####Features +- repo/ubuntu: allow to change uca repo name + +####Maintenance +- implement acceptance tests +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- acceptance: use common bits from puppet-openstack-integration +- fix RSpec 3.x syntax +- initial msync run for all Puppet OpenStack modules + ##2015-07-08 - 6.0.0 ###Summary diff --git a/openstack_extras/README.md b/openstack_extras/README.md index ec351fcc2..7f86a4068 100644 --- a/openstack_extras/README.md +++ b/openstack_extras/README.md @@ -1,7 +1,7 @@ openstack_extras ============ -6.0.0 - 2015.1.0 - Kilo +7.0.0 - 2015.2.0 - Liberty #### Table of Contents diff --git a/openstack_extras/Rakefile b/openstack_extras/Rakefile index ed79bead4..9692ffdd2 100644 --- a/openstack_extras/Rakefile +++ b/openstack_extras/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/openstack_extras/metadata.json b/openstack_extras/metadata.json index 979433203..8d18f496f 100644 --- a/openstack_extras/metadata.json +++ b/openstack_extras/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-openstack_extras", - "version": "6.0.0", + "version": "7.0.0", "author": "OpenStack Contributors", "summary": "Puppet OpenStack Extras Module", "license": "Apache-2.0", diff --git a/openstack_extras/spec/spec_helper_acceptance.rb b/openstack_extras/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/openstack_extras/spec/spec_helper_acceptance.rb +++ b/openstack_extras/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/swift/CHANGELOG.md b/swift/CHANGELOG.md index 101173bda..b3bad2a0e 100644 --- a/swift/CHANGELOG.md +++ b/swift/CHANGELOG.md @@ -1,3 +1,43 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes +- remove tenant parameter from keystone_user + +####Features +- add tag to package and service resources +- add swift::config class +- reflect provider change in puppet-openstacklib +- keystone/auth: make service description configurable +- add support for swift-object-expirer service +- drop useless comment in authtoken.conf.erb +- improve File resources idempotency +- proxy: ceilometer httpd support +- stop managing file modes +- add support for DLO configuration +- warn that object storage parameter mount_check changes next release +- provide means to disable log_requests in config templates +- add incoming/outgoing chmod params to storage/all +- rely on autorequire for config resource ordering +- add tempauth middleware options +- add tempurl middleware options +- config resources applied after config template + +####Bugfixes +- fix swift.conf / Swift_config ordering +- make sure Facter is only executed on agent +- add a blank line to the beginning of each filter + +####Maintenance +- initial msync run for all Puppet OpenStack modules +- spec: Enable webmock connect to IPv4 link-local +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- acceptance: use common bits from puppet-openstack-integration +- fix rspec 3.x syntax + ##2015-10-10 - 6.1.0 ###Summary diff --git a/swift/README.md b/swift/README.md index 79b539e62..e53475953 100644 --- a/swift/README.md +++ b/swift/README.md @@ -1,7 +1,7 @@ swift ======= -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2 - Liberty #### Table of Contents diff --git a/swift/Rakefile b/swift/Rakefile index ed79bead4..9692ffdd2 100644 --- a/swift/Rakefile +++ b/swift/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/swift/metadata.json b/swift/metadata.json index df0a36317..1cb32028c 100644 --- a/swift/metadata.json +++ b/swift/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-swift", - "version": "6.1.0", + "version": "7.0.0", "author": "Puppet Labs and OpenStack Contributors", "summary": "Puppet module for OpenStack Swift", "license": "Apache-2.0", @@ -32,7 +32,7 @@ "description": "Installs and configures OpenStack Swift (Object Storage).", "dependencies": [ { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/rsync", "version_requirement": ">=0.2.0 <1.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, { "name": "puppetlabs/xinetd", "version_requirement": ">=1.0.1 <2.0.0" }, diff --git a/swift/spec/spec_helper_acceptance.rb b/swift/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/swift/spec/spec_helper_acceptance.rb +++ b/swift/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/tempest/CHANGELOG.md b/tempest/CHANGELOG.md index 84391c6bc..2cfb78a69 100644 --- a/tempest/CHANGELOG.md +++ b/tempest/CHANGELOG.md @@ -1,3 +1,27 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-compatible major release for OpenStack Liberty. + +####Features +- allow to enable/disable Aodh service +- allow to enable/disable Trove service +- allow to enable/disable Sahara service +- add dashboard parameters +- add basic scenarios options +- allow to optionally git clone tempest +- reflect provider change in puppet-openstacklib + +####Bugfixes +- make sure neutron network is created before Tempest_neutron_net_id_setter +- glance_id_setter: execute after creating Glance image +- fix ocnfiguration for glance/neutron setters + +####Maintenance +- initial msync run for all Puppet OpenStack modules +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check + ##2015-10-10 - 6.1.0 ###Summary diff --git a/tempest/README.markdown b/tempest/README.markdown index 7d2ad4727..42f968f00 100644 --- a/tempest/README.markdown +++ b/tempest/README.markdown @@ -1,7 +1,7 @@ Tempest ======= -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2 - Liberty Module for installing and configuring tempest. diff --git a/tempest/Rakefile b/tempest/Rakefile index ed79bead4..9692ffdd2 100644 --- a/tempest/Rakefile +++ b/tempest/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/tempest/metadata.json b/tempest/metadata.json index 3a306790c..33f95ac9b 100644 --- a/tempest/metadata.json +++ b/tempest/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-tempest", - "version": "6.1.0", + "version": "7.0.0", "author": "OpenStack Contributors", "summary": "Puppet module for OpenStack Tempest", "license": "Apache-2.0", diff --git a/tempest/spec/spec_helper_acceptance.rb b/tempest/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/tempest/spec/spec_helper_acceptance.rb +++ b/tempest/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/tripleo/.fixtures.yml b/tripleo/.fixtures.yml index e3ab8f9cd..e2444d17f 100644 --- a/tripleo/.fixtures.yml +++ b/tripleo/.fixtures.yml @@ -2,5 +2,26 @@ fixtures: repositories: 'firewall': 'git://github.com/puppetlabs/puppetlabs-firewall.git' 'stdlib': 'git://github.com/puppetlabs/puppetlabs-stdlib.git' + 'midonet': + repo: 'git://github.com/midonet/puppet-midonet.git' + ref: 'v2015.06.7' + 'tomcat': + repo: 'git://github.com/puppetlabs/puppetlabs-tomcat.git' + ref: '1.3.2' + 'inifile': + repo: 'git://github.com/puppetlabs/puppetlabs-inifile.git' + ref: '1.4.2' + 'cassandra': + repo: 'git://github.com/midonet/puppet-cassandra.git' + ref: 'v1.1.1' + 'zookeeper': + repo: 'git://github.com/deric/puppet-zookeeper.git' + ref: 'v0.3.9' + 'datacat': + repo: 'git://github.com/richardc/puppet-datacat' + ref: '0.6.2' + 'java': + repo: 'git://github.com/puppetlabs/puppetlabs-java' + ref: '1.4.2' symlinks: "tripleo": "#{source_dir}" diff --git a/tripleo/lib/puppet/parser/functions/extract_id.rb b/tripleo/lib/puppet/parser/functions/extract_id.rb new file mode 100644 index 000000000..61734abfa --- /dev/null +++ b/tripleo/lib/puppet/parser/functions/extract_id.rb @@ -0,0 +1,14 @@ +# Custom function to extract the index from a list. +# The list are a list of hostname, and the index is the n'th +# position of the host in list +module Puppet::Parser::Functions + newfunction(:extract_id, :type => :rvalue) do |argv| + hosts = argv[0] + if hosts.class != Array + hosts = [hosts] + end + hostname = argv[1] + hash = Hash[hosts.map.with_index.to_a] + return hash[hostname].to_i + 1 + end +end diff --git a/tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb b/tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb new file mode 100644 index 000000000..814326e84 --- /dev/null +++ b/tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb @@ -0,0 +1,24 @@ +# Custom function to convert a list of ips to a map +# like {'ip' => xxx.xxx.xxx.xxx }. This function is needed +# because a not-so-good design of the puppet-midonet module +# and we hope to deprecate it soon. + +module Puppet::Parser::Functions + newfunction(:list_to_zookeeper_hash, :type => :rvalue, :doc => <<-EOS + This function returns Zookeper configuration list of hash + EOS + ) do |argv| + zk_list = argv[0] + if zk_list.class != Array + zk_list = [zk_list] + end + result = Array.new + zk_list.each do |zk_ip| + zk_map = Hash.new + zk_map['ip'] = zk_ip + zk_map['port'] = 2181 + result.push(zk_map) + end + return result + end +end diff --git a/tripleo/manifests/cluster/cassandra.pp b/tripleo/manifests/cluster/cassandra.pp new file mode 100644 index 000000000..a810ccd79 --- /dev/null +++ b/tripleo/manifests/cluster/cassandra.pp @@ -0,0 +1,69 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# == Class: tripleo::cluster::cassandra +# +# Deploys a cassandra service that belongs to a cluster. Uses puppet-cassandra +# +# == Parameters: +# +# [*cassandra_servers*] +# (required) All the IP addresses of the cassandra cluster. +# Array of strings value. +# +# [*cassandra_ip*] +# (required) IP address of the current host. +# String value +# +# [*storage_port*] +# (optional) Inter-node cluster communication port. +# Defaults to 7000. +# +# [*ssl_storage_port*] +# (optional) SSL Inter-node cluster communication port. +# Defaults to 7001. +# +# [*client_port*] +# (optional) Cassandra client port. +# Defaults to 9042. +# +# [*client_port_thrift*] +# (optional) Cassandra client port thrift. +# Defaults to 9160. +# +class tripleo::cluster::cassandra( + $cassandra_servers, + $cassandra_ip, + $storage_port = '7000', + $ssl_storage_port = '7001', + $client_port = '9042', + $client_port_thrift = '9160' +) +{ + validate_array($cassandra_servers) + validate_ipv4_address($cassandra_ip) + + class {'::cassandra::run': + seeds => $cassandra_servers, + seed_address => $cassandra_ip, + conf_dir => '/etc/cassandra/default.conf', + pid_dir => '/var/run/cassandra', + service_path => '/sbin', + storage_port => $storage_port, + ssl_storage_port => $ssl_storage_port, + client_port => $client_port, + client_port_thrift => $client_port_thrift + } +} diff --git a/tripleo/manifests/cluster/zookeeper.pp b/tripleo/manifests/cluster/zookeeper.pp new file mode 100644 index 000000000..6f4adbcd8 --- /dev/null +++ b/tripleo/manifests/cluster/zookeeper.pp @@ -0,0 +1,65 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# == Class: tripleo::cluster::zookeeper +# +# Deploys a zookeeper service that belongs to a cluster. Uses deric-zookeeper +# +# == Parameters: +# +# [*zookeeper_server_ips*] +# (required) List of IP addresses of the zookeeper cluster. +# Arrays of strings value. +# +# [*zookeeper_client_ip*] +# (required) IP address of the host where zookeeper will listen IP addresses. +# String (IPv4) value. +# +# [*zookeeper_hostnames*] +# (required) List of hostnames of the zookeeper cluster. The hostname of the +# node will be used to define the ID of the zookeeper configuration +# Array of strings value. +# + +class tripleo::cluster::zookeeper( + $zookeeper_server_ips, + $zookeeper_client_ip, + $zookeeper_hostnames +) +{ + validate_array($zookeeper_server_ips) + validate_ipv4_address($zookeeper_client_ip) + validate_array($zookeeper_hostnames) + + # TODO(devvesa) Zookeeper package should provide these paths, + # remove this lines as soon as it will. + file {['/usr/lib', '/usr/lib/zookeeper', '/usr/lib/zookeeper/bin/']: + ensure => directory + } + + file {'/usr/lib/zookeeper/bin/zkEnv.sh': + ensure => link, + target => '/usr/libexec/zkEnv.sh' + } + + class {'::zookeeper': + servers => $zookeeper_server_ips, + client_ip => $zookeeper_client_ip, + id => extract_id($zookeeper_hostnames, $::hostname), + cfg_dir => '/etc/zookeeper/conf', + } + + File['/usr/lib/zookeeper/bin/zkEnv.sh'] -> Class['::zookeeper'] +} diff --git a/tripleo/manifests/loadbalancer.pp b/tripleo/manifests/loadbalancer.pp index 16a4f80a9..e34b01e4b 100644 --- a/tripleo/manifests/loadbalancer.pp +++ b/tripleo/manifests/loadbalancer.pp @@ -814,6 +814,7 @@ bind => $nova_novnc_bind_opts, options => { 'balance' => 'source', + 'timeout' => [ 'tunnel 1h' ], }, collect_exported => false, } diff --git a/tripleo/manifests/network/midonet/agent.pp b/tripleo/manifests/network/midonet/agent.pp new file mode 100644 index 000000000..2f1da9414 --- /dev/null +++ b/tripleo/manifests/network/midonet/agent.pp @@ -0,0 +1,61 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# == Class: tripleo::network::midonet::agent +# +# Configure the midonet agent +# +# == Parameters: +# +# [*zookeeper_servers*] +# (required) List of IPs of the zookeeper server cluster. It will configure +# the connection using the 2181 port. +# Array of strings value. +# +# [*cassandra_seeds*] +# (required) List of IPs of the cassandra cluster. +# Array of strings value. +# +class tripleo::network::midonet::agent ( + $zookeeper_servers, + $cassandra_seeds +) { + + validate_array($zookeeper_servers) + validate_array($cassandra_seeds) + + # FIXME: This statement should be controlled by hiera on heat templates + # project + # Make sure openvswitch service is not running + service {'openvswitch': + ensure => stopped, + enable => false + } + + exec {'delete datapaths': + command => '/usr/bin/mm-dpctl --delete-dp ovs-system', + path => '/usr/bin:/usr/sbin:/bin', + onlyif => '/usr/bin/mm-dpctl --show-dp ovs-system' + } + + # Configure and run the agent + class {'::midonet::midonet_agent': + zk_servers => list_to_zookeeper_hash($zookeeper_servers), + cassandra_seeds => $cassandra_seeds + } + + Service['openvswitch'] -> Class['::midonet::midonet_agent::run'] + Exec['delete datapaths'] -> Class['::midonet::midonet_agent::run'] +} diff --git a/tripleo/manifests/network/midonet/api.pp b/tripleo/manifests/network/midonet/api.pp new file mode 100644 index 000000000..33b22178a --- /dev/null +++ b/tripleo/manifests/network/midonet/api.pp @@ -0,0 +1,117 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# == Class: tripleo::network::midonet::api +# +# Configure the MidoNet API +# +# == Parameters: +# +# [*zookeeper_servers*] +# (required) List IPs of the zookeeper server cluster. Zookeeper is the +# backend database where MidoNet stores the virtual network topology. +# Array of strings value. +# +# [*vip*] +# (required) Public Virtual IP where the API will be exposed. +# String (IPv4) value. +# +# [*keystone_ip*] +# (required) MidoNet API is registered as an OpenStack service. Provide the +# keystone ip address. +# String (IPv4) value. +# +# [*keystone_admin_token*] +# (required) MidoNet API is registered as an OpenStack service. It needs the +# keystone admin token to perform some admin calls. +# String value. +# +# [*bind_address*] +# (required) MidoNet API uses a Tomcat instance to offer the REST service. The +# ip address where to bind the tomcat service. +# String (IPv4) value. +# +# [*admin_password*] +# (required) OpenStack admin user password. +# String value. +# +# [*keystone_port*] +# (optional) MidoNet API is registered as an OpenStack service. Provide +# the keystone port. +# Defaults to 35357 +# +# [*keystone_tenant_name*] +# (optional) Tenant of the keystone service. +# Defaults to 'admin' +# +# [*admin_user_name*] +# (optional) OpenStack admin user name. +# Defaults to 'admin' +# +# [*admin_tenant_name*] +# (optional). OpenStack admin tenant name. +# Defaults to 'admin' +# + +class tripleo::network::midonet::api( + $zookeeper_servers, + $vip, + $keystone_ip, + $keystone_admin_token, + $bind_address, + $admin_password, + $keystone_port = 35357, + $keystone_tenant_name = 'admin', + $admin_user_name = 'admin', + $admin_tenant_name = 'admin' +) +{ + validate_array($zookeeper_servers) + validate_ip_address($vip) + validate_ip_address($keystone_ip) + validate_ip_address($bind_address) + + # Run Tomcat and MidoNet API + class {'::tomcat': + install_from_source => false + } -> + + package {'midonet-api': + ensure => present + } -> + + class {'::midonet::midonet_api::run': + zk_servers => list_to_zookeeper_hash($zookeeper_servers), + keystone_auth => true, + tomcat_package => 'tomcat', + vtep => false, + api_ip => $vip, + api_port => '8081', + keystone_host => $keystone_ip, + keystone_port => $keystone_port, + keystone_admin_token => $keystone_admin_token, + keystone_tenant_name => $keystone_tenant_name, + catalina_base => '/usr/share/tomcat', + bind_address => $bind_address + } + + # Configure the CLI + class {'::midonet::midonet_cli': + api_endpoint => "http://${vip}:8081/midonet-api", + username => $admin_user_name, + password => $admin_password, + tenant_name => $admin_tenant_name + } +} diff --git a/tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb b/tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb new file mode 100644 index 000000000..4f2eb6c0f --- /dev/null +++ b/tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb @@ -0,0 +1,45 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# Unit tests for the cassandra service + +require 'spec_helper' + +describe 'tripleo::cluster::cassandra' do + + shared_examples_for 'cassandra cluster service' do + + let :params do + { + :cassandra_servers => ['192.168.2.2', '192.168.2.3'], + :cassandra_ip => '192.168.2.2' + } + end + + it 'should configure cassandra' do + is_expected.to contain_class('cassandra::run').with( + :seeds => ['192.168.2.2', '192.168.2.3'], + :seed_address => '192.168.2.2', + :storage_port => '7000', + :ssl_storage_port => '7001', + :client_port => '9042', + :client_port_thrift => '9160' + ) + end + end + + it_configures 'cassandra cluster service' + +end diff --git a/tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb b/tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb new file mode 100644 index 000000000..ed46164a4 --- /dev/null +++ b/tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb @@ -0,0 +1,115 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# Unit tests for the zookeeper service + +require 'spec_helper' + +describe 'tripleo::cluster::zookeeper' do + + let :default_params do + { + :zookeeper_server_ips => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], + :zookeeper_hostnames => ['host1.midonet', 'host2.midonet', 'host3.midonet'] + } + end + + context 'on host1' do + let :facts do + { + :hostname => 'host1.midonet', + :osfamily => 'RedHat', + :operatingsystemmajrelease => 7, + } + end + + let :params do + { + :zookeeper_client_ip => '23.43.2.34' + } + end + + before do + params.merge!(default_params) + end + + it 'should call zookeeper using id==1' do + is_expected.to contain_class('zookeeper').with( + :servers => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], + :client_ip => '23.43.2.34', + :id => 1 + ) + end + + end + + context 'on host2' do + let :facts do + { + :hostname => 'host2.midonet', + :osfamily => 'RedHat', + :operatingsystemmajrelease => 7, + } + end + + let :params do + { + :zookeeper_client_ip => '23.43.2.35' + } + end + + before do + params.merge!(default_params) + end + + it 'should call zookeeper using id==1' do + is_expected.to contain_class('zookeeper').with( + :servers => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], + :client_ip => '23.43.2.35', + :id => 2 + ) + end + end + + context 'on host3' do + let :facts do + { + :hostname => 'host3.midonet', + :osfamily => 'RedHat', + :operatingsystemmajrelease => 7, + } + end + + let :params do + { + :zookeeper_client_ip => '23.43.2.36' + } + end + + before do + params.merge!(default_params) + end + + it 'should call zookeeper using id==1' do + is_expected.to contain_class('zookeeper').with( + :servers => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], + :client_ip => '23.43.2.36', + :id => 3 + ) + end + + end + +end diff --git a/tripleo/spec/classes/tripleo_firewall_spec.rb b/tripleo/spec/classes/tripleo_firewall_spec.rb index c1249b9fa..7d1d1ecb8 100644 --- a/tripleo/spec/classes/tripleo_firewall_spec.rb +++ b/tripleo/spec/classes/tripleo_firewall_spec.rb @@ -105,7 +105,10 @@ context 'on RedHat platforms' do let :facts do - { :osfamily => 'RedHat' } + { + :osfamily => 'RedHat', + :operatingsystemrelease => '7.1', + } end it_configures 'tripleo node' diff --git a/tripleo/spec/classes/tripleo_midonet_agent_spec.rb b/tripleo/spec/classes/tripleo_midonet_agent_spec.rb new file mode 100644 index 000000000..eb3abfe85 --- /dev/null +++ b/tripleo/spec/classes/tripleo_midonet_agent_spec.rb @@ -0,0 +1,62 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# Unit tests for the midonet agent + +require 'spec_helper' + +describe 'tripleo::network::midonet::agent' do + + let :facts do + { + :hostname => 'host2.midonet', + :osfamily => 'RedHat', + :operatingsystem => 'CentOS', + :operatingsystemrelease => '7.1', + :operatingsystemmajrelease => 7, + } + end + + shared_examples_for 'midonet agent test' do + + let :params do + { + :zookeeper_servers => ['192.168.2.2', '192.168.2.3'], + :cassandra_seeds => ['192.168.2.2', '192.168.2.3'] + } + end + + it 'should stop openvswitch' do + is_expected.to contain_service('openvswitch').with( + :ensure => 'stopped', + :enable => false + ) + end + + it 'should run the agent with a list of maps' do + is_expected.to contain_class('midonet::midonet_agent').with( + :zk_servers => [{'ip' => '192.168.2.2', + 'port' => 2181}, + {'ip' => '192.168.2.3', + 'port' => 2181}], + :cassandra_seeds => ['192.168.2.2','192.168.2.3'] + ) + end + end + + it_configures 'midonet agent test' + + +end diff --git a/tripleo/spec/classes/tripleo_midonet_api_spec.rb b/tripleo/spec/classes/tripleo_midonet_api_spec.rb new file mode 100644 index 000000000..4b4729494 --- /dev/null +++ b/tripleo/spec/classes/tripleo_midonet_api_spec.rb @@ -0,0 +1,72 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# Unit tests for the midonet api + +require 'spec_helper' + +describe 'tripleo::network::midonet::api' do + + let :facts do + { + :augeasversion => '1.0.0' + } + end + + shared_examples_for 'midonet api test' do + + let :params do + { + :zookeeper_servers => ['192.168.2.1', '192.168.2.2'], + :vip => '192.23.0.2', + :keystone_ip => '192.23.0.2', + :keystone_admin_token => 'admin_token', + :admin_password => 'admin_password', + :bind_address => '192.23.0.65' + } + end + + it 'should call api configuration' do + is_expected.to contain_class('midonet::midonet_api::run').with( + :zk_servers => [{'ip' => '192.168.2.1', 'port' => 2181}, + {'ip' => '192.168.2.2', 'port' => 2181}], + :keystone_auth => true, + :tomcat_package => 'tomcat', + :vtep => false, + :api_ip => '192.23.0.2', + :api_port => '8081', + :keystone_host => '192.23.0.2', + :keystone_port => 35357, + :keystone_admin_token => 'admin_token', + :keystone_tenant_name => 'admin', + :catalina_base => '/usr/share/tomcat', + :bind_address => '192.23.0.65' + ) + end + + it 'should install the cli' do + is_expected.to contain_class('midonet::midonet_cli').with( + :api_endpoint => 'http://192.23.0.2:8081/midonet-api', + :username => 'admin', + :password => 'admin_password', + :tenant_name => 'admin' + ) + end + + end + + it_configures 'midonet api test' + +end diff --git a/tripleo/spec/spec_helper.rb b/tripleo/spec/spec_helper.rb index 700be6a27..5cf9642eb 100644 --- a/tripleo/spec/spec_helper.rb +++ b/tripleo/spec/spec_helper.rb @@ -2,9 +2,13 @@ require 'shared_examples' require 'webmock/rspec' +fixture_path = File.expand_path(File.join(__FILE__, '..', 'fixtures')) + RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' c.alias_it_should_behave_like_to :it_raises, 'raises' + c.module_path = File.join(fixture_path, 'modules') + c.manifest_dir = File.join(fixture_path, 'manifests') c.default_facts = { :kernel => 'Linux', diff --git a/trove/CHANGELOG.md b/trove/CHANGELOG.md index 4767b5458..ab8713226 100644 --- a/trove/CHANGELOG.md +++ b/trove/CHANGELOG.md @@ -1,3 +1,33 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-compatible major release for OpenStack Liberty. + +####Features +- add tag to package and service resources +- add trove::config class +- reflect provider change in puppet-openstacklib +- introduce trove::quota class +- introduce use_guestagent_template option +- make taskmanager_queue option configurable +- add api ratelimit options +- add region and resource url related options +- add default_neutron_networks in trove::taskmanager +- complete qpid support +- keystone/auth: make service description configurable + +####Bugfixes +- fix rabbit_userid parameter +- fix default value of guestagent_config_file option + +####Maintenance +- initial msync run for all Puppet OpenStack modules +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- acceptance: use common bits from puppet-openstack-integration +- acceptance: enable debug & verbosity for OpenStack logs +- fix rspec 3.x syntax + ##2015-10-10 - 6.1.0 ###Summary diff --git a/trove/README.md b/trove/README.md index fe6836b84..eecbb2d68 100644 --- a/trove/README.md +++ b/trove/README.md @@ -1,7 +1,7 @@ puppet-trove ============= -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2 - Liberty #### Table of Contents diff --git a/trove/Rakefile b/trove/Rakefile index ed79bead4..9692ffdd2 100644 --- a/trove/Rakefile +++ b/trove/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/trove/manifests/api.pp b/trove/manifests/api.pp index 9333283a4..401247186 100644 --- a/trove/manifests/api.pp +++ b/trove/manifests/api.pp @@ -370,6 +370,9 @@ } if $::trove::rpc_backend == 'trove.openstack.common.rpc.impl_qpid' or $::trove::rpc_backend == 'qpid'{ + + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + trove_config { 'oslo_messaging_qpid/qpid_hostname': value => $::trove::qpid_hostname; 'oslo_messaging_qpid/qpid_port': value => $::trove::qpid_port; diff --git a/trove/manifests/conductor.pp b/trove/manifests/conductor.pp index 0d3b5adee..0f9b4a52f 100644 --- a/trove/manifests/conductor.pp +++ b/trove/manifests/conductor.pp @@ -165,6 +165,9 @@ } if $::trove::rpc_backend == 'trove.openstack.common.rpc.impl_qpid' or $::trove::rpc_backend == 'qpid'{ + + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + trove_conductor_config { 'oslo_messaging_qpid/qpid_hostname': value => $::trove::qpid_hostname; 'oslo_messaging_qpid/qpid_port': value => $::trove::qpid_port; diff --git a/trove/manifests/guestagent.pp b/trove/manifests/guestagent.pp index 2d803a6a7..65d0740e2 100644 --- a/trove/manifests/guestagent.pp +++ b/trove/manifests/guestagent.pp @@ -162,6 +162,9 @@ } if $::trove::rpc_backend == 'trove.openstack.common.rpc.impl_qpid' or $::trove::rpc_backend == 'qpid'{ + + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + trove_guestagent_config { 'oslo_messaging_qpid/qpid_hostname': value => $::trove::qpid_hostname; 'oslo_messaging_qpid/qpid_port': value => $::trove::qpid_port; diff --git a/trove/manifests/taskmanager.pp b/trove/manifests/taskmanager.pp index cd4adedb9..cb42167f8 100644 --- a/trove/manifests/taskmanager.pp +++ b/trove/manifests/taskmanager.pp @@ -215,6 +215,9 @@ } if $::trove::rpc_backend == 'trove.openstack.common.rpc.impl_qpid' or $::trove::rpc_backend == 'qpid'{ + + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + trove_taskmanager_config { 'oslo_messaging_qpid/qpid_hostname': value => $::trove::qpid_hostname; 'oslo_messaging_qpid/qpid_port': value => $::trove::qpid_port; diff --git a/trove/metadata.json b/trove/metadata.json index e8cd79be8..a728fd3df 100644 --- a/trove/metadata.json +++ b/trove/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-trove", - "version": "6.1.0", + "version": "7.0.0", "author": "eNovance and OpenStack Contributors", "summary": "Puppet module for OpenStack Trove", "license": "Apache-2.0", @@ -32,8 +32,8 @@ "description": "Installs and configures OpenStack Trove (Database service).", "dependencies": [ { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" } + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" } ] } diff --git a/trove/spec/spec_helper_acceptance.rb b/trove/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/trove/spec/spec_helper_acceptance.rb +++ b/trove/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/vswitch/CHANGELOG.md b/vswitch/CHANGELOG.md index 9011ada23..be1eb0881 100644 --- a/vswitch/CHANGELOG.md +++ b/vswitch/CHANGELOG.md @@ -1,3 +1,23 @@ +##2015-11-25 - 3.0.0 +###Summary + +This is a major release for OpenStack Liberty but contains no API-breaking +changes. + + +####Features +- support for FreeBSD + +###Bugfixes +- explicitly say that ovs_redhat parent is ovs +- add require ovs_redhat.rb to ovs_redhat_el6.rb + +####Maintenance +- acceptance: use common bits from puppet-openstack-integration +- remove class_parameter_defaults puppet-lint check +- fix RSpec 3.x syntax +- initial msync run for all Puppet OpenStack modules + ##2015-10-15 - 2.1.0 ###Summary diff --git a/vswitch/README.md b/vswitch/README.md index 0b2c4f371..24ac3be3c 100644 --- a/vswitch/README.md +++ b/vswitch/README.md @@ -1,7 +1,7 @@ VSwitch ======= -2.1.0 - 2015.1 - Kilo +3.0.0 - 2015.2 - Liberty A Puppet module providing things for vSwitches. At the moment OVS is the only one I've added but please feel free to contribute new providers through diff --git a/vswitch/metadata.json b/vswitch/metadata.json index 60bf7ce9d..a0e7ddb7f 100644 --- a/vswitch/metadata.json +++ b/vswitch/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-vswitch", - "version": "2.1.0", + "version": "3.0.0", "author": "Endre Karlson, Dan Bode and OpenStack Contributors", "license": "Apache-2.0", "source": "git://github.com/openstack/puppet-vswitch.git", @@ -27,6 +27,10 @@ { "operatingsystem": "Ubuntu", "operatingsystemrelease": [ "12.04", "14.04" ] + }, + { + "operatingsystem":"FreeBSD", + "operatingsystemrelease": [ "10.0", "11.0" ] } ], "requirements": [ diff --git a/vswitch/spec/spec_helper_acceptance.rb b/vswitch/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/vswitch/spec/spec_helper_acceptance.rb +++ b/vswitch/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" From 5d5b419b50fb2cb2bf32dd6b13decbf4b2267e59 Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Fri, 4 Dec 2015 12:19:43 -0500 Subject: [PATCH 02/35] Add puppet-fluentd module. This is to be used as part of the operational toolchain. This module update commit was generated by Bade. For more info please check https://github.com/paramite/bade This commit is setting modules to following state: fluentd - initial commit: 828fc99cd6c631e8b34f47b8fdcfa3d30dd8daa2 Change-Id: I7091bfcdeadd33dd5a343f04bc625668930bc7e2 --- Puppetfile | 4 + fluentd/.fixtures.yml | 8 + fluentd/.gemfile | 16 ++ fluentd/.gitignore | 7 + fluentd/.travis.yml | 36 +++ fluentd/CHANGELOG | 23 ++ fluentd/Gemfile | 21 ++ fluentd/LICENSE | 202 +++++++++++++++ fluentd/Modulefile | 13 + fluentd/README.md | 195 +++++++++++++++ fluentd/Rakefile | 2 + fluentd/files/packages.treasure-data.com.key | 18 ++ .../lib/puppet/provider/package/fluentgem.rb | 137 ++++++++++ fluentd/manifests/config.pp | 19 ++ fluentd/manifests/configfile.pp | 15 ++ fluentd/manifests/filter.pp | 35 +++ fluentd/manifests/forest_match.pp | 15 ++ fluentd/manifests/init.pp | 17 ++ fluentd/manifests/install_plugin.pp | 40 +++ fluentd/manifests/install_plugin/file.pp | 24 ++ fluentd/manifests/install_plugin/gem.pp | 21 ++ fluentd/manifests/install_repo/apt.pp | 24 ++ fluentd/manifests/install_repo/yum.pp | 19 ++ fluentd/manifests/match.pp | 17 ++ fluentd/manifests/packages.pp | 50 ++++ fluentd/manifests/params.pp | 12 + fluentd/manifests/service.pp | 12 + fluentd/manifests/source.pp | 17 ++ fluentd/spec/classes/fluentd_config_spec.rb | 58 +++++ fluentd/spec/classes/fluentd_packages_spec.rb | 70 ++++++ fluentd/spec/classes/fluentd_service_spec.rb | 79 ++++++ fluentd/spec/classes/fluentd_spec.rb | 233 ++++++++++++++++++ fluentd/spec/defines/fluent_filter_spec.rb | 39 +++ .../spec/defines/fluentd_configfile_spec.rb | 30 +++ fluentd/spec/defines/fluentd_match_spec.rb | 76 ++++++ .../fluentgem/gem-list-single-package | 4 + .../fluentgem/line-with-1.8.5-warning | 14 ++ fluentd/spec/spec.opts | 4 + fluentd/spec/spec_helper.rb | 2 + fluentd/spec/unit/provider/fluentgem_spec.rb | 176 +++++++++++++ fluentd/spec_helper.rb | 1 + fluentd/templates/filter.erb | 30 +++ fluentd/templates/forest_match.erb | 21 ++ fluentd/templates/match.erb | 33 +++ fluentd/templates/source.erb | 16 ++ fluentd/templates/td-agent.conf.erb | 2 + 46 files changed, 1907 insertions(+) create mode 100644 fluentd/.fixtures.yml create mode 100644 fluentd/.gemfile create mode 100644 fluentd/.gitignore create mode 100644 fluentd/.travis.yml create mode 100644 fluentd/CHANGELOG create mode 100644 fluentd/Gemfile create mode 100644 fluentd/LICENSE create mode 100644 fluentd/Modulefile create mode 100644 fluentd/README.md create mode 100644 fluentd/Rakefile create mode 100644 fluentd/files/packages.treasure-data.com.key create mode 100644 fluentd/lib/puppet/provider/package/fluentgem.rb create mode 100644 fluentd/manifests/config.pp create mode 100644 fluentd/manifests/configfile.pp create mode 100644 fluentd/manifests/filter.pp create mode 100644 fluentd/manifests/forest_match.pp create mode 100644 fluentd/manifests/init.pp create mode 100644 fluentd/manifests/install_plugin.pp create mode 100644 fluentd/manifests/install_plugin/file.pp create mode 100644 fluentd/manifests/install_plugin/gem.pp create mode 100644 fluentd/manifests/install_repo/apt.pp create mode 100644 fluentd/manifests/install_repo/yum.pp create mode 100644 fluentd/manifests/match.pp create mode 100644 fluentd/manifests/packages.pp create mode 100644 fluentd/manifests/params.pp create mode 100644 fluentd/manifests/service.pp create mode 100644 fluentd/manifests/source.pp create mode 100644 fluentd/spec/classes/fluentd_config_spec.rb create mode 100644 fluentd/spec/classes/fluentd_packages_spec.rb create mode 100644 fluentd/spec/classes/fluentd_service_spec.rb create mode 100644 fluentd/spec/classes/fluentd_spec.rb create mode 100644 fluentd/spec/defines/fluent_filter_spec.rb create mode 100644 fluentd/spec/defines/fluentd_configfile_spec.rb create mode 100644 fluentd/spec/defines/fluentd_match_spec.rb create mode 100644 fluentd/spec/fixtures/unit/provider/fluentgem/gem-list-single-package create mode 100644 fluentd/spec/fixtures/unit/provider/fluentgem/line-with-1.8.5-warning create mode 100644 fluentd/spec/spec.opts create mode 100644 fluentd/spec/spec_helper.rb create mode 100644 fluentd/spec/unit/provider/fluentgem_spec.rb create mode 100644 fluentd/spec_helper.rb create mode 100644 fluentd/templates/filter.erb create mode 100644 fluentd/templates/forest_match.erb create mode 100644 fluentd/templates/match.erb create mode 100644 fluentd/templates/source.erb create mode 100644 fluentd/templates/td-agent.conf.erb diff --git a/Puppetfile b/Puppetfile index e22f40bef..002e657d8 100644 --- a/Puppetfile +++ b/Puppetfile @@ -58,6 +58,10 @@ mod 'firewall', :commit => '1b6cc9192150d9521cc70301d0452daf189a63f7', :git => 'https://github.com/puppetlabs/puppetlabs-firewall.git' +mod 'fluentd', + :commit => '828fc99cd6c631e8b34f47b8fdcfa3d30dd8daa2', + :git => 'https://github.com/mmz-srf/puppet-fluentd.git' + mod 'galera', :commit => 'a2ecf273aef9ef9193bba35c235fb7b8b821a0c3', :git => 'https://github.com/redhat-openstack/puppet-galera.git' diff --git a/fluentd/.fixtures.yml b/fluentd/.fixtures.yml new file mode 100644 index 000000000..779d63a00 --- /dev/null +++ b/fluentd/.fixtures.yml @@ -0,0 +1,8 @@ +fixtures: + repositories: + apt: "https://github.com/puppetlabs/puppetlabs-apt.git" + concat: "https://github.com/puppetlabs/puppetlabs-concat.git" + stdlib: "https://github.com/puppetlabs/puppetlabs-stdlib.git" + symlinks: + "fluentd": "#{source_dir}" + diff --git a/fluentd/.gemfile b/fluentd/.gemfile new file mode 100644 index 000000000..e7ea9dbb9 --- /dev/null +++ b/fluentd/.gemfile @@ -0,0 +1,16 @@ +source 'https://rubygems.org' + +group :development, :test do + gem 'rake', :require => false + gem 'rspec-puppet', :require => false + gem 'puppetlabs_spec_helper', :require => false + gem 'rspec-system', :require => false + gem 'rspec-system-puppet', :require => false + gem 'rspec-system-serverspec', :require => false +end + +if puppetversion = ENV['PUPPET_GEM_VERSION'] + gem 'puppet', puppetversion, :require => false +else + gem 'puppet', :require => false +end diff --git a/fluentd/.gitignore b/fluentd/.gitignore new file mode 100644 index 000000000..e7beff00f --- /dev/null +++ b/fluentd/.gitignore @@ -0,0 +1,7 @@ +pkg/ +Gemfile.lock +vendor/ +spec/fixtures/modules/ +.vagrant/ +.bundle/ +coverage/ diff --git a/fluentd/.travis.yml b/fluentd/.travis.yml new file mode 100644 index 000000000..0a8b188f4 --- /dev/null +++ b/fluentd/.travis.yml @@ -0,0 +1,36 @@ +--- +branches: + only: + - master +language: ruby +bundler_args: --without development +script: bundle exec rake spec SPEC_OPTS='--format documentation' +rvm: + - 1.9.3 + - 2.0.0 +gemfile: .gemfile +env: + matrix: + - PUPPET_GEM_VERSION="~> 2.7.0" + - PUPPET_GEM_VERSION="~> 3.1.0" + - PUPPET_GEM_VERSION="~> 3.2.0" + - PUPPET_GEM_VERSION="~> 3.3.0" + - PUPPET_GEM_VERSION="~> 3.4.0" + - PUPPET_GEM_VERSION="~> 3.5.0" + - PUPPET_GEM_VERSION="~> 3.6.0" +matrix: + exclude: + - rvm: 1.9.3 + env: PUPPET_GEM_VERSION="~> 2.7.0" + gemfile: .gemfile + - rvm: 1.9.3 + env: PUPPET_GEM_VERSION="~> 3.1.0" + gemfile: .gemfile + - rvm: 2.0.0 + env: PUPPET_GEM_VERSION="~> 2.7.0" + gemfile: .gemfile + - rvm: 2.0.0 + env: PUPPET_GEM_VERSION="~> 3.1.0" + gemfile: .gemfile +notifications: + email: gianni.carafa@srf.ch diff --git a/fluentd/CHANGELOG b/fluentd/CHANGELOG new file mode 100644 index 000000000..33cc56afb --- /dev/null +++ b/fluentd/CHANGELOG @@ -0,0 +1,23 @@ +## puppet-fluentd changelog + +Release notes for the srfmmz/puppet-fluentd module. + +------------------------------------------ + +#### 0.1.4 - 2014/08/05 + +* FEATURE added filter functionality (by softecspa) +* FIX configfile syntax fix for td-agent v2 (by Molezeem) +* CHANGE changed the plugin installation to fluentgem provider (by saleksee) + +#### 0.1.3 - 2014/07/06 + +Please update to get the new repository of Treasuredata. + +Detailed Changes: + +* FEATURE : added time_format for source configuration +* CHANGE : Added new treasuredata repositories. + - for more info see here : https://groups.google.com/forum/#!topic/fluentd/46FnWIoLTrU +* CHANGE : Restart td-agent when config changed +* FEATURE : Startetd this changelog diff --git a/fluentd/Gemfile b/fluentd/Gemfile new file mode 100644 index 000000000..e30fed0ea --- /dev/null +++ b/fluentd/Gemfile @@ -0,0 +1,21 @@ +# Gemfile to run bundler command +# usagen: bundler exec rake spec + +source 'https://rubygems.org' + +puppetversion = ENV.key?('PUPPET_VERSION') ? "~> #{ENV['PUPPET_VERSION']}" : ['>= 3.2.1'] +gem 'puppet', puppetversion + +if puppetversion =~ /^3/ + ## rspec-hiera-puppet is puppet 3 only + gem 'rspec-hiera-puppet', '>=1.0.0' +end + +facterversion = ENV.key?('FACTER_VERSION') ? "~> #{ENV['FACTER_VERSION']}" : ['>= 1.7.1'] + +gem 'facter', facterversion + +gem 'rake' +gem 'rspec' +gem 'rspec-puppet', '>=0.1.6' +gem 'puppetlabs_spec_helper', '>=0.4.1' diff --git a/fluentd/LICENSE b/fluentd/LICENSE new file mode 100644 index 000000000..e06d20818 --- /dev/null +++ b/fluentd/LICENSE @@ -0,0 +1,202 @@ +Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "{}" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright {yyyy} {name of copyright owner} + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + diff --git a/fluentd/Modulefile b/fluentd/Modulefile new file mode 100644 index 000000000..520e5e410 --- /dev/null +++ b/fluentd/Modulefile @@ -0,0 +1,13 @@ +name 'srf-fluentd' +version '0.1.4' + +dependency 'puppetlabs/apt', '>= 1.4.0' +dependency 'puppetlabs/concat', '>= 1.0.0' +dependency 'puppetlabs/stdlib', '>= 4.1.0' + +summary 'fluentd module' +description 'Manages Fluentd installation and configuration with Puppet using the td-agent.' +source 'https://github.com/mmz-srf/puppet-fluentd.git' +project_page 'https://github.com/mmz-srf/puppet-fluentd' +author 'Gianni Carafa' +license 'Apache License, Version 2.0' diff --git a/fluentd/README.md b/fluentd/README.md new file mode 100644 index 000000000..f46c78617 --- /dev/null +++ b/fluentd/README.md @@ -0,0 +1,195 @@ +puppet-fluentd +============== + +[![Build Status](https://travis-ci.org/mmz-srf/puppet-fluentd.png?branch=master)](https://travis-ci.org/mmz-srf/puppet-fluentd) + +Manage Fluentd installation, configuration and Plugin-management with Puppet using the td-agent. + +## Supported Operating Systems +- Debian (tested on Debian 7.5) +- Ubuntu +- Redhat +- CentOS (tested on CentOS 6.4) + +## Used Modules +- apt: "https://github.com/puppetlabs/puppetlabs-apt.git" (Only for Debian) +- concat: "https://github.com/puppetlabs/puppetlabs-concat.git" +- stdlib: "https://github.com/puppetlabs/puppetlabs-stdlib.git" + +## Contributing +* Fork it +* Create a feature branch (`git checkout -b my-new-feature`) +* Run rspec tests (`bundle exec rake spec` or `rake spec`) +* Commit your changes (`git commit -am 'Added some feature'`) +* Push to the branch (`git push origin my-new-feature`) +* Create new Pull Request + +### Branches in this module +- **master** tested and working. Latest Version on http://forge.puppetlabs.com +- **developmemt** includes the newewst features. works too, mostly. + +## Configuration +How to configure a Agent to send data to a centralised Fluentd-Server + +### Install a Plugin +Install your fluentd plugin. (check here for the right pluginname : http://fluentd.org/plugin/ ). + +You can choose from a file or gem based instalation. +``` + include ::fluentd + + fluentd::install_plugin { 'elasticsearch': + plugin_type => 'gem', + plugin_name => 'fluent-plugin-elasticsearch', + } +``` + +### Create a Agent +The Agent watches over your logfiles and sends its content to the Collector. +``` + include ::fluentd + + fluentd::configfile { 'apache': } + fluentd::source { 'apache_main': + configfile => 'apache' + type => 'tail', + format => 'apache2', + tag => 'apache.access_log', + config => { + 'path' => '/var/log/apache2/access.log', + 'pos_file' => '/var/tmp/fluentd.pos', + }, + notify => Class['fluentd::service'] + } + + fluentd::configfile { 'syslog': } + fluentd::source { 'syslog_main': + configfile => 'syslog', + type => 'tail', + format => 'syslog', + tag => 'system.syslog', + config => { + 'path' => '/var/log/syslog', + 'pos_file' => '/tmp/td-agent.syslog.pos', + }, + notify => Class['fluentd::service'] + } + + fluentd::configfile { 'forward': } + fluentd::match { 'forward_main': + configfile => 'forward' + pattern => '**', + type => 'forward', + servers => [ + {'host' => 'PUT_YOUR_HOST_HERE', 'port' => '24224'} + ], + notify => Class['fluentd::service'] + } +``` +#### creates on the Agent side following files : +``` +/etc/td-agent/ + ├── config.d + │   ├── apache.conf + │   ├── syslog.conf + │   └── forward.conf + ├── ... + ... +``` + +### Create a Collector +The Collector collects all the data from the Agents. He now stores the data in files, Elasticsearch or elsewhere. +``` + include ::fluentd + + fluentd::configfile { 'collector': } + fluentd::source { 'collector_main': + configfile => 'collector', + type => 'forward', + notify => Class['fluentd::service'] + } + + fluentd::match { 'collector_main': + configfile => 'collector', + pattern => '**', + type => 'elasticsearch', + config => { + 'logstash_format' => true, + }, + notify => Class['fluentd::service'] + } + + # receive syslog messages on port 5140 + # all rsyslog daemons on the clients sends theire messages to 5140 + fluentd::configfile { 'rsyslog': } + fluentd::source { 'rsyslog_main': + configfile => 'rsyslog', + type => 'syslog', + tag => 'system.local', + config => { + 'port' => '5140', + 'bind' => '0.0.0.0', + }, + notify => Class['fluentd::service'] + } +``` + +#### creates on the Collectors side following files : +``` +/etc/td-agent/ + ├── config.d + │   └── collector.conf + ├── ... + ... +``` + +### copy ouput to multiple stores +```` + $logger=[ { 'host' => 'logger-sample01', 'port' => '24224'}, + { 'host' => 'logger-example01', 'port' => '24224', 'standby' => ''} ] + + fluentd::match { 'forward_to_logger': + configfile => 'sample_tail', + pattern => 'alocal', + type => 'copy', + config => [ + { + 'type' => 'forward', + 'send_timeout' => '60s', + 'recover_wait' => '10s', + 'heartbeat_interval' => '1s', + 'phi_threshold' => 8, + 'hard_timeout' => '60s', + 'flush_interval' => '5s', + 'servers' => $logger, + }, + { + 'type' => 'stdout', + 'output_type' => 'json', + } + ], + notify => Class['fluentd::service'] + } + +``` + +### add a filter +``` + fluentd::configfile { 'myfilter': } + fluentd::filter { 'myfilter_main': + configfile => 'myfilter', + pattern => '**', + type => 'grep', + input_key => 'key', + regexp => '/*.foo.*/', + exclude => 'baar', + output_tag => 'mytag', + add_tag_prefix => 'pre_', + remove_tag_prefix => 'remove_', + add_tag_suffix => '_after', + remove_tag_suffix => '_remove', + config => { + 'customvalue' => true, + } + } +``` diff --git a/fluentd/Rakefile b/fluentd/Rakefile new file mode 100644 index 000000000..bbfcc8b9c --- /dev/null +++ b/fluentd/Rakefile @@ -0,0 +1,2 @@ +require 'rubygems' +require 'puppetlabs_spec_helper/rake_tasks' \ No newline at end of file diff --git a/fluentd/files/packages.treasure-data.com.key b/fluentd/files/packages.treasure-data.com.key new file mode 100644 index 000000000..c86c35489 --- /dev/null +++ b/fluentd/files/packages.treasure-data.com.key @@ -0,0 +1,18 @@ +-----BEGIN PGP PUBLIC KEY BLOCK----- +Version: GnuPG v1.4.5 (GNU/Linux) + +mQGiBFKuy6MRBACJccrkWHfYSKSbchD4ywYLGkIKITSLzEXrlpy4Sp6Mr9G1OGIv +buIUIxksnev6h6wlFgbFYTLngpod+Jn4DpcGgXqzZ3lhOY1fvAaQ4SJ4RUELm1JJ +SEWE0uYHuoI4+TrLtod3U02ETD+Lf6LWO3IfU1AYsA467tDYpVXR0GHeAwCgqU/V +M+ypNOYFHVs1/aX83wdg0K8D+wf+tcvTHb/i2lND5DVfTWtC54WYNIG6QO8DgGdu +EYregcMKRRNM4nSfUISzCgrjnpLGjvB9NKONCrimDiuz4TH7KDcEsKGA+zTzJLCy +mcE6tKEHV5yS7cK0inim4nxAKFB70306CkhHyL2TrB42ppNFJwG1igO4BJ+55b9X +Dd03A/9o6ONFle6khG+UE82WYTayy2pAshKTSLnOlB7CIxwt3SrzFXTuNosajVgZ +S0EIB8c6SjFN6P1jHjd7PYOmBxd2xptERTg8CJMoTUI+P7LypWuvmagIdEwe571d +ZMQ26HNuHg7/IMg+FQxhl/DFU6Aq+2CBJ3q8gO8Us4/UYc1PhrRUVHJlYXN1cmUg +RGF0YSwgSW5jIChUcmVhc3VyZSBBZ2VudCBPZmZpY2lhbCBTaWduaW5nIGtleSkg +PHN1cHBvcnRAdHJlYXN1cmUtZGF0YS5jb20+iGAEExECACAFAlKuy6MCGwMGCwkI +BwMCBBUCCAMEFgIDAQIeAQIXgAAKCRAQk9tFoS4gb3HgAKCBCDNT/cWzkKRQxvnL +LiGvPetVSACeLLKnt2twfiPc9ZEA/X8MKkPjWqc= +=e5Fe +-----END PGP PUBLIC KEY BLOCK----- diff --git a/fluentd/lib/puppet/provider/package/fluentgem.rb b/fluentd/lib/puppet/provider/package/fluentgem.rb new file mode 100644 index 000000000..9c5165481 --- /dev/null +++ b/fluentd/lib/puppet/provider/package/fluentgem.rb @@ -0,0 +1,137 @@ +require 'puppet/provider/package' +require 'uri' + +# Ruby gems support. +Puppet::Type.type(:package).provide :fluentgem, :parent => Puppet::Provider::Package do + desc "Install gem via fluent-gem (included by td-agent). Ruby Gem support. If a URL is passed via `source`, then that URL is used as the + remote gem repository; if a source is present but is not a valid URL, it will be + interpreted as the path to a local gem file. If source is not present at all, + the gem will be installed from the default gem repositories. + + This provider supports the `install_options` attribute, which allows command-line flags to be passed to the gem command. + These options should be specified as a string (e.g. '--flag'), a hash (e.g. {'--flag' => 'value'}), + or an array where each element is either a string or a hash." + + has_feature :versionable, :install_options + + ENV['PATH'] = "#{ENV['PATH']}:/usr/lib64/fluent/ruby/bin:/usr/lib/fluent/ruby/bin" + + commands :gemcmd => "fluent-gem" + + def self.gemlist(options) + gem_list_command = [command(:gemcmd), "list"] + + if options[:local] + gem_list_command << "--local" + else + gem_list_command << "--remote" + end + if options[:source] + gem_list_command << "--source" << options[:source] + end + if name = options[:justme] + gem_list_command << "^" + name + "$" + end + + begin + list = execute(gem_list_command).lines. + map {|set| gemsplit(set) }. + reject {|x| x.nil? } + rescue Puppet::ExecutionFailure => detail + raise Puppet::Error, "Could not list gems: #{detail}", detail.backtrace + end + + if options[:justme] + return list.shift + else + return list + end + end + + def self.gemsplit(desc) + # `gem list` when output console has a line like: + # *** LOCAL GEMS *** + # but when it's not to the console that line + # and all blank lines are stripped + # so we don't need to check for them + + if desc =~ /^(\S+)\s+\((.+)\)/ + name = $1 + versions = $2.split(/,\s*/) + { + :name => name, + :ensure => versions.map{|v| v.split[0]}, + :provider => :gem + } + else + Puppet.warning "Could not match #{desc}" unless desc.chomp.empty? + nil + end + end + + def self.instances(justme = false) + gemlist(:local => true).collect do |hash| + new(hash) + end + end + + def install(useversion = true) + command = [command(:gemcmd), "install"] + command << "-v" << resource[:ensure] if (! resource[:ensure].is_a? Symbol) and useversion + + if source = resource[:source] + begin + uri = URI.parse(source) + rescue => detail + self.fail Puppet::Error, "Invalid source '#{uri}': #{detail}", detail + end + + case uri.scheme + when nil + # no URI scheme => interpret the source as a local file + command << source + when /file/i + command << uri.path + when 'puppet' + # we don't support puppet:// URLs (yet) + raise Puppet::Error.new("puppet:// URLs are not supported as gem sources") + else + # interpret it as a gem repository + command << "--source" << "#{source}" << resource[:name] + end + else + command << "--no-rdoc" << "--no-ri" << resource[:name] + end + + command += install_options if resource[:install_options] + + output = execute(command) + # Apparently some stupid gem versions don't exit non-0 on failure + self.fail "Could not install: #{output.chomp}" if output.include?("ERROR") + end + + def latest + # This always gets the latest version available. + gemlist_options = {:justme => resource[:name]} + gemlist_options.merge!({:source => resource[:source]}) unless resource[:source].nil? + hash = self.class.gemlist(gemlist_options) + + hash[:ensure][0] + end + + def query + self.class.gemlist(:justme => resource[:name], :local => true) + end + + def uninstall + gemcmd "uninstall", "-x", "-a", resource[:name] + end + + def update + self.install(false) + end + + def install_options + join_options(resource[:install_options]) + end +end diff --git a/fluentd/manifests/config.pp b/fluentd/manifests/config.pp new file mode 100644 index 000000000..f9d62526c --- /dev/null +++ b/fluentd/manifests/config.pp @@ -0,0 +1,19 @@ +# Class: fluentd::config() +# +# +class fluentd::config() { + file { '/etc/td-agent/td-agent.conf' : + ensure => file, + owner => 'root', + group => 'root', + content => template('fluentd/td-agent.conf.erb'), + notify => Class['fluentd::service'], + } + + file {'/etc/td-agent/config.d': + ensure => 'directory', + owner => 'td-agent', + group => 'td-agent', + mode => '0750', + } +} \ No newline at end of file diff --git a/fluentd/manifests/configfile.pp b/fluentd/manifests/configfile.pp new file mode 100644 index 000000000..509cf4eef --- /dev/null +++ b/fluentd/manifests/configfile.pp @@ -0,0 +1,15 @@ +# == definition fluentd::configfile +define fluentd::configfile { + $source_conf = "/etc/td-agent/config.d/${title}.conf" + if ! defined(Class['fluentd']) { + fail('You must include the fluentd base class before using any fluentd defined resources') + } + concat{$source_conf: + owner => 'td-agent', + group => 'td-agent', + mode => '0644', + require => Class['Fluentd::Packages'], + notify => Class['Fluentd::Service'], + } +} + diff --git a/fluentd/manifests/filter.pp b/fluentd/manifests/filter.pp new file mode 100644 index 000000000..1d55517d7 --- /dev/null +++ b/fluentd/manifests/filter.pp @@ -0,0 +1,35 @@ +# == definition fluentd::filter +define fluentd::filter ( + $configfile, + $pattern, + $type = 'grep', + $input_key = '', + $regexp = '', + $exclude = '', + $config = {}, + $output_tag = '', + $add_tag_prefix = '', + $remove_tag_prefix = '', + $add_tag_suffix = '', + $remove_tag_suffix = '', +) { + + if ($type == 'grep') { + if !defined(Fluentd::Install_plugin['fluent-plugin-grep']) { + fluentd::install_plugin{'fluent-plugin-grep': + plugin_type => 'gem' + } + } + } + + if ((($regexp != '') or ($exclude != '')) and ($input_key == '')) or (($input_key != '') and ($regexp == '') and ($exclude == '')) { + fail ('regexp, exlude and input_key must be used in conjuction') + } + + concat::fragment { 'filter': + target => "/etc/td-agent/config.d/${configfile}.conf", + require => Class['Fluentd::Packages'], + content => template('fluentd/filter.erb'), + } + +} diff --git a/fluentd/manifests/forest_match.pp b/fluentd/manifests/forest_match.pp new file mode 100644 index 000000000..09ca9a996 --- /dev/null +++ b/fluentd/manifests/forest_match.pp @@ -0,0 +1,15 @@ +# == definition fluentd::forest_match +define fluentd::forest_match ( + $configfile, + $type, + $pattern, + $config = {}, + $servers = [], +) { + + concat::fragment { "match_${title}": + target => "/etc/td-agent/config.d/${configfile}.conf", + require => Package["${fluentd::package_name}"], + content => template('fluentd/forest_match.erb'), + } +} diff --git a/fluentd/manifests/init.pp b/fluentd/manifests/init.pp new file mode 100644 index 000000000..f0e095634 --- /dev/null +++ b/fluentd/manifests/init.pp @@ -0,0 +1,17 @@ +# == class fluentd +class fluentd ( + $version = '1', + $package_name = $fluentd::params::package_name, + $install_repo = $fluentd::params::install_repo, + $package_ensure = $fluentd::params::package_ensure, + $service_enable = $fluentd::params::service_enable, + $service_ensure = $fluentd::params::service_ensure +) inherits fluentd::params { + class{'fluentd::packages': } + class{'fluentd::config': } + class{'fluentd::service': } + + validate_bool($install_repo, $service_enable) + + Class['Fluentd::Packages'] -> Class['Fluentd::Config'] -> Class['Fluentd::Service'] +} diff --git a/fluentd/manifests/install_plugin.pp b/fluentd/manifests/install_plugin.pp new file mode 100644 index 000000000..a395b7fde --- /dev/null +++ b/fluentd/manifests/install_plugin.pp @@ -0,0 +1,40 @@ +# == fluentd::install_plugin +# +# install a plugin with either /usr/lib/fluent/ruby/bin/fluent-gem or a file +# +# you need to sepecify wich one of theese options plus the name. +# +# Parameters: +# the name of this ressource reflects the either the filename of the plugin, which must +# be copied to fluentd/files or the name of the gem +# +# plugin_type: specify "file" to copy that file to /etc/fluentd/plugins +# specify "gem" to try to install the plugin with fluent-gem +# +# ensure: "present"(default) or "absent", install or uninstall a plugin +# +define fluentd::install_plugin ( + $plugin_type, + $ensure = 'present', + $plugin_name = $name, +) { + case $plugin_type { + 'file': { + fluentd::install_plugin::file { + [$plugin_name]: + ensure => $ensure, + require => Class['Fluentd::Packages'] + } + } + 'gem': { + fluentd::install_plugin::gem { + [$plugin_name]: + ensure => $ensure, + require => Class['Fluentd::Packages'] + } + } + default: { + fail("plugin_type => ${plugin_type} is currently not supportet by this module") + } + } +} diff --git a/fluentd/manifests/install_plugin/file.pp b/fluentd/manifests/install_plugin/file.pp new file mode 100644 index 000000000..47d3ab155 --- /dev/null +++ b/fluentd/manifests/install_plugin/file.pp @@ -0,0 +1,24 @@ +# == fluentd::install_plugin::file +# +# install a plugin with by copying a file to /etc/td-agent/plugins +# +# Parameters: +# the name of this ressource reflects the filename of the plugin, which must +# be copied to fluentd/files +# +# ensure: "present"(default) or "absent", install or uninstall a plugin +# +define fluentd::install_plugin::file ( + $ensure = 'present', + $plugin_name = $name, +) { + file { + "/etc/td-agent/plugin/${plugin_name}": + ensure => $ensure, + owner => td-agent, + group => td-agent, + mode => '0640', + source => "puppet:///fluentd/plugins/${plugin_name}", + notify => Service["${::fluentd::service_name}"]; + } +} diff --git a/fluentd/manifests/install_plugin/gem.pp b/fluentd/manifests/install_plugin/gem.pp new file mode 100644 index 000000000..bd8e14508 --- /dev/null +++ b/fluentd/manifests/install_plugin/gem.pp @@ -0,0 +1,21 @@ +# == fluentd::install_plugin::gem +# +# install a plugin with /usr/lib/fluent/ruby/bin/fluent-gem +# +# Parameters: +# the name of this ressource reflects the name of the gem +# +# ensure: "present"(default) or "absent", install or uninstall a plugin +# +define fluentd::install_plugin::gem ( + $ensure = 'latest', + $plugin_name = $name, +) { + + package { $plugin_name: + ensure => $ensure, + provider => 'fluentgem', + notify => Service["${fluentd::service_name}"]; + } + +} diff --git a/fluentd/manifests/install_repo/apt.pp b/fluentd/manifests/install_repo/apt.pp new file mode 100644 index 000000000..e2eadbfc1 --- /dev/null +++ b/fluentd/manifests/install_repo/apt.pp @@ -0,0 +1,24 @@ +##apt.pp + +# Class: fluentd::install_repo::apt () +# +# +class fluentd::install_repo::apt () { + + apt::source { 'treasure-data': + location => "http://packages.treasuredata.com/debian", + release => "lucid", + repos => "contrib", + include_src => false, + } + + file { '/tmp/packages.treasure-data.com.key': + ensure => file, + source => 'puppet:///modules/fluentd/packages.treasure-data.com.key' + }-> + exec { "import gpg key Treasure Data": + command => "/bin/cat /tmp/packages.treasure-data.com.key | apt-key add -", + unless => "/usr/bin/apt-key list | grep -q 'Treasure Data'", + notify => Class['::apt::update'], + } +} diff --git a/fluentd/manifests/install_repo/yum.pp b/fluentd/manifests/install_repo/yum.pp new file mode 100644 index 000000000..790781465 --- /dev/null +++ b/fluentd/manifests/install_repo/yum.pp @@ -0,0 +1,19 @@ +# yum.pp + +# Class: fluentd::install_repo::yum () +# +# +class fluentd::install_repo::yum ( + $key = $fluentd::params::yum_key_url, + ) { + + # Sorry for the different naming of the Repository between debian and redhat. + # But I dont want rename it to avoid a duplication. + yumrepo { 'treasuredata': + descr => 'Treasure Data', + baseurl => 'http://packages.treasuredata.com/redhat/$basearch', + gpgkey => 'http://packages.treasuredata.com/redhat/RPM-GPG-KEY-td-agent', + gpgcheck => 1, + } + +} diff --git a/fluentd/manifests/match.pp b/fluentd/manifests/match.pp new file mode 100644 index 000000000..49b3f2d93 --- /dev/null +++ b/fluentd/manifests/match.pp @@ -0,0 +1,17 @@ +# == definition fluentd::match +define fluentd::match ( + $configfile, + $type, + $pattern, + $config = {}, + $servers = [], + $order = undef, +) { + + concat::fragment { "match_${title}": + target => "/etc/td-agent/config.d/${configfile}.conf", + require => Package["${fluentd::package_name}"], + content => template('fluentd/match.erb'), + order => $order, + } +} diff --git a/fluentd/manifests/packages.pp b/fluentd/manifests/packages.pp new file mode 100644 index 000000000..e48793579 --- /dev/null +++ b/fluentd/manifests/packages.pp @@ -0,0 +1,50 @@ +# == class fluentd::packages +class fluentd::packages ( + $package_name = $fluentd::package_name, + $install_repo = $fluentd::install_repo, + $package_ensure = $fluentd::package_ensure +){ + if $install_repo { + case $::osfamily { + 'redhat': { + class{'fluentd::install_repo::yum': + before => Package[$package_name], + } + } + 'debian': { + class{'fluentd::install_repo::apt': + before => Package[$package_name], + } + } + default: { + fail("Unsupported osfamily ${::osfamily}") + } + } + } + package { "$package_name": + ensure => $package_ensure + } + +# extra bits... why this is required isn't quite clear. + case $::osfamily { + 'debian': { + package{[ + 'libxslt1.1', + 'libyaml-0-2', + ]: + before => Package[$package_name], + ensure => $package_ensure + } + exec {'add user td-agent to group adm': + provider => shell, + unless => '/bin/grep -q "adm\S*td-agent" /etc/group', + command => '/usr/sbin/usermod -aG adm td-agent', + subscribe => Package[$package_name], + } + } + default: { + info("No required fluentd::packages extra bits for ${::osfamily}") + } + } + +} diff --git a/fluentd/manifests/params.pp b/fluentd/manifests/params.pp new file mode 100644 index 000000000..aed3b9a31 --- /dev/null +++ b/fluentd/manifests/params.pp @@ -0,0 +1,12 @@ +# Class: fluentd::params +# +# +class fluentd::params { + $package_name = 'td-agent' + $package_ensure = 'installed' + $install_repo = true + $service_ensure = 'running' + $service_enable = true + $service_name = 'td-agent' + $yum_key_url = "http://packages.treasure-data.com/redhat/RPM-GPG-KEY-td-agent" +} \ No newline at end of file diff --git a/fluentd/manifests/service.pp b/fluentd/manifests/service.pp new file mode 100644 index 000000000..a539f0978 --- /dev/null +++ b/fluentd/manifests/service.pp @@ -0,0 +1,12 @@ +# == class fluentd::service +class fluentd::service ( + $service_ensure = $fluentd::service_ensure, + $service_enable = $fluentd::service_enable, +) { + include fluentd::params + service {"${fluentd::params::service_name}": + ensure => $service_ensure, + enable => $service_enable, + hasstatus => true + } +} diff --git a/fluentd/manifests/source.pp b/fluentd/manifests/source.pp new file mode 100644 index 000000000..b71738474 --- /dev/null +++ b/fluentd/manifests/source.pp @@ -0,0 +1,17 @@ +# == definition fluentd::source +define fluentd::source ( + $configfile, + $type, + $tag = false, + $format = false, + $time_format = false, + $config = {}, + $pattern = [], +) { + + concat::fragment { "source_${title}": + target => "/etc/td-agent/config.d/${configfile}.conf", + require => Package["${fluentd::package_name}"], + content => template('fluentd/source.erb'), + } +} diff --git a/fluentd/spec/classes/fluentd_config_spec.rb b/fluentd/spec/classes/fluentd_config_spec.rb new file mode 100644 index 000000000..cb744b569 --- /dev/null +++ b/fluentd/spec/classes/fluentd_config_spec.rb @@ -0,0 +1,58 @@ +#!/usr/bin/env rspec +require 'spec_helper' + +describe 'fluentd::config', :type => :class do + context "On a Debian OS, config files should be written" do + let :facts do + { + :osfamily => 'Debian', + :operatingsystemrelease => '7', + :concat_basedir => '/tmp', + :lsbdistid => 'Debian', + } + end + + it "/etc/td-agent/td-agent.conf should be in place" do + should contain_file("/etc/td-agent/td-agent.conf").with( + 'ensure' => 'file', + 'owner' => 'root', + 'group' => 'root', + 'notify' => 'Class[Fluentd::Service]' + ).with_content(/^# Include.*config.d.*/m) + end + it "/etc/td-agent/config.d is created" do + should contain_file("/etc/td-agent/config.d").with( + 'ensure' => 'directory', + 'owner' => 'td-agent', + 'group' => 'td-agent', + 'mode' => '0750' + ) + end + end + context "On a Redhat OS, config files should be written" do + let :facts do + { + :osfamily => 'RedHat', + :operatingsystemrelease => '6.5', + :concat_basedir => '/tmp', + :lsbdistid => 'CentOS', + } + end + it "/etc/td-agent/td-agent.conf should be in place" do + should contain_file("/etc/td-agent/td-agent.conf").with( + 'ensure' => 'file', + 'owner' => 'root', + 'group' => 'root', + 'notify' => 'Class[Fluentd::Service]' + ).with_content(/^# Include.*config.d.*/m) + end + it "/etc/td-agent/config.d is created" do + should contain_file("/etc/td-agent/config.d").with( + 'ensure' => 'directory', + 'owner' => 'td-agent', + 'group' => 'td-agent', + 'mode' => '0750' + ) + end + end +end diff --git a/fluentd/spec/classes/fluentd_packages_spec.rb b/fluentd/spec/classes/fluentd_packages_spec.rb new file mode 100644 index 000000000..bad912abb --- /dev/null +++ b/fluentd/spec/classes/fluentd_packages_spec.rb @@ -0,0 +1,70 @@ +#!/usr/bin/env rspec +require 'spec_helper' + +describe 'fluentd::packages', :type => :class do + let (:params) {{:package_name => 'td-agent', :package_ensure => 'installed'}} + context "On a Debian OS" do + let :facts do + { + :osfamily => 'Debian', + :operatingsystemrelease => '7', + :lsbdistid => 'Debian', + } + end + + context "with install_repo=>true" do + let(:params) { {:install_repo => true} } + it do + should contain_apt__source("treasure-data").with( + 'location' => 'http://packages.treasuredata.com/debian' + ) + end + end + it { should contain_package("libxslt1.1").with( + 'ensure' => 'installed' + ) + } + it { should contain_package("libyaml-0-2").with( + 'ensure' => 'installed' + ) + } + it { should contain_package("td-agent").with( + 'ensure' => 'installed' + ) + } + end + + context "On a RedHat/Centos OS" do + + let :params do + { + :package_name => 'td-agent', + :package_ensure => 'running', + } + end + + let :facts do + { + :osfamily => 'Redhat', + } + end + + it { should contain_class('fluentd::packages')} + + context "with install_repo=>true" do + let(:params) { {:install_repo => true} } + it do + should contain_yumrepo('treasuredata').with( + 'baseurl' => 'http://packages.treasuredata.com/redhat/$basearch', + 'gpgkey' => 'http://packages.treasuredata.com/redhat/RPM-GPG-KEY-td-agent', + 'gpgcheck' => 1 + ) + end + end + + it { should contain_package("td-agent").with( + 'ensure' => 'running' + ) + } + end +end diff --git a/fluentd/spec/classes/fluentd_service_spec.rb b/fluentd/spec/classes/fluentd_service_spec.rb new file mode 100644 index 000000000..c9ec2bbaf --- /dev/null +++ b/fluentd/spec/classes/fluentd_service_spec.rb @@ -0,0 +1,79 @@ +#!/usr/bin/env rspec +require 'spec_helper' + +describe 'fluentd::service', :type => :class do + context "On a Debian OS" do + let :facts do + { + :osfamily => 'Debian', + :operatingsystemrelease => '7', + :concat_basedir => '/tmp', + :lsbdistid => 'Debian', + } + end + context "td-agent running and enabled" do + let (:params) {{:service_ensure => 'running', :service_enable => true}} + it { should contain_service("td-agent").with( + 'ensure' => 'running', + 'enable' => 'true', + 'hasstatus' => 'true' + ) + } + end + context "td-agent stopped but enabled" do + let (:params) {{:service_ensure => 'stopped', :service_enable => true}} + it { should contain_service("td-agent").with( + 'ensure' => 'stopped', + 'enable' => 'true', + 'hasstatus' => 'true' + ) + } + end + context "td-agent stopped and disabled" do + let (:params) {{:service_ensure => 'stopped', :service_enable => false}} + it { should contain_service("td-agent").with( + 'ensure' => 'stopped', + 'enable' => 'false', + 'hasstatus' => 'true' + ) + } + end + end + context "On a Redhat OS" do + let :facts do + { + :osfamily => 'RedHat', + :operatingsystemrelease => '6.5', + :concat_basedir => '/tmp', + :lsbdistid => 'CentOs', + } + end + context "td-agent running and enabled" do + let (:params) {{:service_ensure => 'running', :service_enable => true}} + it { should contain_service("td-agent").with( + 'ensure' => 'running', + 'enable' => 'true', + 'hasstatus' => 'true' + ) + } + end + context "td-agent stopped but enabled" do + let (:params) {{:service_ensure => 'stopped', :service_enable => true}} + it { should contain_service("td-agent").with( + 'ensure' => 'stopped', + 'enable' => 'true', + 'hasstatus' => 'true' + ) + } + end + context "td-agent stopped and disabled" do + let (:params) {{:service_ensure => 'stopped', :service_enable => false}} + it { should contain_service("td-agent").with( + 'ensure' => 'stopped', + 'enable' => 'false', + 'hasstatus' => 'true' + ) + } + end + end +end diff --git a/fluentd/spec/classes/fluentd_spec.rb b/fluentd/spec/classes/fluentd_spec.rb new file mode 100644 index 000000000..0d5c7cb4d --- /dev/null +++ b/fluentd/spec/classes/fluentd_spec.rb @@ -0,0 +1,233 @@ +#!/usr/bin/env rspec +require 'spec_helper' + +describe 'fluentd', :type => :class do + context "On a Debian OS " do + let :facts do + { + :osfamily => 'Debian', + :operatingsystemrelease => '7', + :concat_basedir => '/tmp', + :lsbdistid => 'Debian', + } + end + context "Debian with repo installed" do + let (:params) {{ + :package_name => 'td-agent', + :package_ensure => 'installed', + :install_repo => true, + :service_enable => true, + :service_ensure => 'running' + }} + it "has fluentd::packages" do + should contain_class('fluentd::packages') + end + it "has fluentd::config class" do + should contain_class('fluentd::config') + end + it "has fluentd::service class" do + should contain_class('fluentd::service') + end + it "the apt repo file exists" do + should contain_apt__source("treasure-data").with( + 'location' => 'http://packages.treasuredata.com/debian' + ) + end + it "/etc/td-agent/td-agent.conf should be in place" do + should contain_file("/etc/td-agent/td-agent.conf").with( + 'ensure' => 'file', + 'owner' => 'root', + 'group' => 'root', + 'notify' => 'Class[Fluentd::Service]' + ).with_content(/^# Include.*config.d.*/m) + end + it "/etc/td-agent/config.d is created" do + should contain_file("/etc/td-agent/config.d").with( + 'ensure' => 'directory', + 'owner' => 'td-agent', + 'group' => 'td-agent', + 'mode' => '0750' + ) + end + it "td-agent package should be installed" do + should contain_package("td-agent").with( + 'ensure' => 'installed' + ) + end + it "td-agent is running" do + should contain_service("td-agent").with( + 'ensure' => 'running', + 'enable' => 'true', + 'hasstatus' => 'true' + ) + end + end + context "Debian without repo installed" do + let (:params) {{ + :package_name => 'td-agent', + :package_ensure => 'installed', + :install_repo => false, + :service_enable => true, + :service_ensure => 'running' + }} + it "has fluentd::packages" do + should contain_class('fluentd::packages') + end + it "has fluentd::config class" do + should contain_class('fluentd::config') + end + it "has fluentd::service class" do + should contain_class('fluentd::service') + end + it "the apt repo file DOES NOT exists" do + should_not contain_apt__source("treasure-data").with( + 'location' => 'http://packages.treasure-data.com/debian' + ) + end + it "/etc/td-agent/td-agent.conf should be in place" do + should contain_file("/etc/td-agent/td-agent.conf").with( + 'ensure' => 'file', + 'owner' => 'root', + 'group' => 'root', + 'notify' => 'Class[Fluentd::Service]' + ).with_content(/^# Include.*config.d.*/m) + end + it "/etc/td-agent/config.d is created" do + should contain_file("/etc/td-agent/config.d").with( + 'ensure' => 'directory', + 'owner' => 'td-agent', + 'group' => 'td-agent', + 'mode' => '0750' + ) + end + it "td-agent package should be installed" do + should contain_package("td-agent").with( + 'ensure' => 'installed' + ) + end + it "td-agent is running" do + should contain_service("td-agent").with( + 'ensure' => 'running', + 'enable' => 'true', + 'hasstatus' => 'true' + ) + end + end + end + context "On a Redhat-like OS with repo installed" do + let (:params) {{ + :package_name => 'td-agent', + :package_ensure => 'installed', + :install_repo => true, + :service_enable => true, + :service_ensure => 'running' + }} + let :facts do + { + :osfamily => 'RedHat', + :operatingsystemrelease => '6.5', + :concat_basedir => '/tmp', + :lsbdistid => 'CentOs', + } + end + + it "has fluentd::packages" do + should contain_class('fluentd::packages') + end + it "has fluentd::config class" do + should contain_class('fluentd::config') + end + it "has fluentd::service class" do + should contain_class('fluentd::service') + end + it "the yum repo file exists" do + should contain_yumrepo('treasuredata') + end + it "/etc/td-agent/td-agent.conf should be in place" do + should contain_file("/etc/td-agent/td-agent.conf").with( + 'ensure' => 'file', + 'owner' => 'root', + 'group' => 'root', + 'notify' => 'Class[Fluentd::Service]' + ).with_content(/^# Include.*config.d.*/m) + end + it "/etc/td-agent/config.d is created" do + should contain_file("/etc/td-agent/config.d").with( + 'ensure' => 'directory', + 'owner' => 'td-agent', + 'group' => 'td-agent', + 'mode' => '0750' + ) + end + it "td-agent package should be installed" do + should contain_package("td-agent").with( + 'ensure' => 'installed' + ) + end + it "td-agent is running" do + should contain_service("td-agent").with( + 'ensure' => 'running', + 'enable' => 'true', + 'hasstatus' => 'true' + ) + end + end + context "On a Redhat-like OS without repo installed" do + let (:params) {{ + :package_name => 'td-agent', + :package_ensure => 'installed', + :install_repo => false, + :service_enable => true, + :service_ensure => 'running' + }} + let :facts do + { + :osfamily => 'RedHat', + :operatingsystemrelease => '6.5', + :concat_basedir => '/tmp', + :lsbdistid => 'CentOs', + } + end + + it "has fluentd::packages" do + should contain_class('fluentd::packages') + end + it "has fluentd::config class" do + should contain_class('fluentd::config') + end + it "has fluentd::service class" do + should contain_class('fluentd::service') + end + it "the yum repo file DOES NOT exists" do + should_not contain_file('/etc/yum.repos.d/td.repo').with_content(/^name=TreasureData$/) + end + it "/etc/td-agent/td-agent.conf should be in place" do + should contain_file("/etc/td-agent/td-agent.conf").with( + 'ensure' => 'file', + 'owner' => 'root', + 'group' => 'root', + 'notify' => 'Class[Fluentd::Service]' + ).with_content(/^# Include.*config.d.*/m) + end + it "/etc/td-agent/config.d is created" do + should contain_file("/etc/td-agent/config.d").with( + 'ensure' => 'directory', + 'owner' => 'td-agent', + 'group' => 'td-agent', + 'mode' => '0750' + ) + end + it "td-agent package should be installed" do + should contain_package("td-agent").with( + 'ensure' => 'installed' + ) + end + it "td-agent is running" do + should contain_service("td-agent").with( + 'ensure' => 'running', + 'enable' => 'true', + 'hasstatus' => 'true' + ) + end + end +end diff --git a/fluentd/spec/defines/fluent_filter_spec.rb b/fluentd/spec/defines/fluent_filter_spec.rb new file mode 100644 index 000000000..39827cbdf --- /dev/null +++ b/fluentd/spec/defines/fluent_filter_spec.rb @@ -0,0 +1,39 @@ +#!/usr/bin/env rspec +require 'spec_helper' + +describe 'fluentd::filter' do + let(:title) {'bar'} + + let (:facts) {{ + :osfamily => 'Debian', + :concat_basedir => '/dne', + :lsbdistid => 'Debian', + }} + + context "when type is grep" do + let(:params) {{ + :configfile => 'foo', + :pattern => 'baz', + :type => 'grep', + :input_key => 'input_key', + :regexp => '/regex/', + :exclude => 'exclude', + :config => {}, + :output_tag => 'output_tag', + :add_tag_prefix => 'add_tag_prefix', + :remove_tag_prefix => 'remove_tag_prefix', + :add_tag_suffix => 'add_tag_suffix', + :remove_tag_suffix => 'remove_tag_suffix', + }} + + it "should install the grep plugin" do + should contain_fluentd__install_plugin('fluent-plugin-grep') + end + + + it "should create matcher single segment" do + should contain_concat__fragment('filter').with_content(/.*type.*input_key.*regexp.*exclude.*output_tag.*add_tag_prefix.*remove_tag_prefix.*add_tag_suffix.*remove_tag_suffix.*<\/match>/m) + end + end + +end \ No newline at end of file diff --git a/fluentd/spec/defines/fluentd_configfile_spec.rb b/fluentd/spec/defines/fluentd_configfile_spec.rb new file mode 100644 index 000000000..6f3bb2675 --- /dev/null +++ b/fluentd/spec/defines/fluentd_configfile_spec.rb @@ -0,0 +1,30 @@ +#!/usr/bin/env rspec +require 'spec_helper' + +describe 'fluentd::configfile', :type => :define do + let :pre_condition do + 'include fluentd' + end + ['Debian','RedHat'].each do |osfam| + context "When on an #{osfam} system" do + let (:facts) {{ + :osfamily => osfam, + :concat_basedir => '/dne', + :lsbdistid => 'Debian', # Concatlib needs this value. Works for RedHat too. + }} + context 'when fed no parameters' do + let (:title) { 'MyBaconBringsAllTheBoysToTheYard'} + context 'provides a stub config' do + it { should contain_class('fluentd') } + it { should contain_concat("/etc/td-agent/config.d/#{title}.conf").with( + 'owner' => 'td-agent', + 'group' => 'td-agent', + 'mode' => '0644', + 'require' => 'Class[Fluentd::Packages]' + ) + } + end + end + end + end +end \ No newline at end of file diff --git a/fluentd/spec/defines/fluentd_match_spec.rb b/fluentd/spec/defines/fluentd_match_spec.rb new file mode 100644 index 000000000..2c459add6 --- /dev/null +++ b/fluentd/spec/defines/fluentd_match_spec.rb @@ -0,0 +1,76 @@ +#!/usr/bin/env rspec +require 'spec_helper' + +describe 'fluentd::match' do + let(:title) {'bar'} + + let (:facts) {{ + :osfamily => 'Debian', + :concat_basedir => '/dne', + :lsbdistid => 'Debian', + }} + + context "when no servers or out_copy" do + let(:params) {{ + :configfile => 'foo', + :pattern => 'baz', + :type => 'file', + :config => { + 'time_slice_wait' => '10m', + 'compress' => 'gzip', + } + }} + + it "should create matcher single segment" do + should contain_concat__fragment('match_bar').with_content(/.*type.*file.*compress.*gzip.*time_slice_wait.*10m.*<\/match>/m) + should_not contain_concat__fragment('match_bar').with_content(/server/) + should_not contain_concat__fragment('match_bar').with_content(/store/) + end + end + + context "when servers but no out_copy" do + let(:params) {{ + :configfile => 'foo', + :pattern => 'baz', + :type => 'file', + :servers => [{ 'host' => 'kelis', 'port' => '24224'}, { 'host' => 'bossy', 'port' => '24224'}], + :config => { + 'time_slice_wait' => '10m', + 'compress' => 'gzip', + } + }} + + it "should create matcher with server" do + should contain_concat__fragment('match_bar').with_content(/.*type.*file.*compress.*gzip.*time_slice_wait.*10m.*.*host kelis.*port.*24224.*<\/server>.*.*host.*bossy.*port.*24224.*<\/server>.*<\/match>/m) + should contain_concat__fragment('match_bar').with_content(/server/) + should_not contain_concat__fragment('match_bar').with_content(/store/) + end + end + + context "when out_copy" do + let(:params) {{ + :configfile => 'foo', + :pattern => 'baz', + :type => 'copy', + :config => [ + { + 'type' => 'file', + 'compress' => 'gzip', + 'servers' => [{ 'host' => 'kelis', 'port' => '24224'}, { 'host' => 'bossy', 'port' => '24224'}], + }, + { + 'type' => 'mongo', + 'database' => 'dummy', + } + ] + }} + + it "should create matcher with server" do + should contain_concat__fragment('match_bar').with_content(/.*type.*copy.*.*compress.*gzip.*.*host.*kelis.*port.*24224.*<\/server>.*.*host.*bossy.*port.*24224.*<\/server>.*type.*file.*<\/store>.*.*database.*dummy.*type.*mongo.*<\/store>.*<\/match>/m) + end + end + + +end + + diff --git a/fluentd/spec/fixtures/unit/provider/fluentgem/gem-list-single-package b/fluentd/spec/fixtures/unit/provider/fluentgem/gem-list-single-package new file mode 100644 index 000000000..41576e3e6 --- /dev/null +++ b/fluentd/spec/fixtures/unit/provider/fluentgem/gem-list-single-package @@ -0,0 +1,4 @@ + +*** REMOTE GEMS *** + +bundler (1.6.2) diff --git a/fluentd/spec/fixtures/unit/provider/fluentgem/line-with-1.8.5-warning b/fluentd/spec/fixtures/unit/provider/fluentgem/line-with-1.8.5-warning new file mode 100644 index 000000000..07a27e7b8 --- /dev/null +++ b/fluentd/spec/fixtures/unit/provider/fluentgem/line-with-1.8.5-warning @@ -0,0 +1,14 @@ +/home/jenkins/.rvm/gems/ruby-1.8.5-p231@global/gems/rubygems-bundler-0.9.0/lib/rubygems-bundler/regenerate_binstubs_command.rb:34: warning: parenthesize argument(s) for future version + +*** LOCAL GEMS *** + +columnize (0.3.2) +diff-lcs (1.1.3) +metaclass (0.0.1) +mocha (0.10.5) +rake (0.8.7) +rspec-core (2.9.0) +rspec-expectations (2.9.1) +rspec-mocks (2.9.0) +rubygems-bundler (0.9.0) +rvm (1.11.3.3) diff --git a/fluentd/spec/spec.opts b/fluentd/spec/spec.opts new file mode 100644 index 000000000..de653df4b --- /dev/null +++ b/fluentd/spec/spec.opts @@ -0,0 +1,4 @@ +--format s +--colour +--loadby mtime +--backtrace diff --git a/fluentd/spec/spec_helper.rb b/fluentd/spec/spec_helper.rb new file mode 100644 index 000000000..479ed6922 --- /dev/null +++ b/fluentd/spec/spec_helper.rb @@ -0,0 +1,2 @@ +require 'rubygems' +require 'puppetlabs_spec_helper/module_spec_helper' \ No newline at end of file diff --git a/fluentd/spec/unit/provider/fluentgem_spec.rb b/fluentd/spec/unit/provider/fluentgem_spec.rb new file mode 100644 index 000000000..1e324ae94 --- /dev/null +++ b/fluentd/spec/unit/provider/fluentgem_spec.rb @@ -0,0 +1,176 @@ +#! /usr/bin/env ruby +require 'spec_helper' + +provider_class = Puppet::Type.type(:package).provider(:gem) + +describe provider_class do + let(:resource) do + Puppet::Type.type(:package).new( + :name => 'myresource', + :ensure => :installed + ) + end + + let(:provider) do + provider = provider_class.new + provider.resource = resource + provider + end + + before :each do + resource.provider = provider + end + + describe "when installing" do + it "should use the path to the gem" do + provider_class.stubs(:command).with(:gemcmd).returns "/my/gem" + provider.expects(:execute).with { |args| args[0] == "/my/gem" }.returns "" + provider.install + end + + it "should specify that the gem is being installed" do + provider.expects(:execute).with { |args| args[1] == "install" }.returns "" + provider.install + end + + it "should specify that documentation should not be included" do + provider.expects(:execute).with { |args| args[2] == "--no-rdoc" }.returns "" + provider.install + end + + it "should specify that RI should not be included" do + provider.expects(:execute).with { |args| args[3] == "--no-ri" }.returns "" + provider.install + end + + it "should specify the package name" do + provider.expects(:execute).with { |args| args[4] == "myresource" }.returns "" + provider.install + end + + describe "when a source is specified" do + describe "as a normal file" do + it "should use the file name instead of the gem name" do + resource[:source] = "/my/file" + provider.expects(:execute).with { |args| args[2] == "/my/file" }.returns "" + provider.install + end + end + describe "as a file url" do + it "should use the file name instead of the gem name" do + resource[:source] = "file:///my/file" + provider.expects(:execute).with { |args| args[2] == "/my/file" }.returns "" + provider.install + end + end + describe "as a puppet url" do + it "should fail" do + resource[:source] = "puppet://my/file" + lambda { provider.install }.should raise_error(Puppet::Error) + end + end + describe "as a non-file and non-puppet url" do + it "should treat the source as a gem repository" do + resource[:source] = "http://host/my/file" + provider.expects(:execute).with { |args| args[2..4] == ["--source", "http://host/my/file", "myresource"] }.returns "" + provider.install + end + end + describe "with an invalid uri" do + it "should fail" do + URI.expects(:parse).raises(ArgumentError) + resource[:source] = "http:::::uppet:/:/my/file" + lambda { provider.install }.should raise_error(Puppet::Error) + end + end + end + end + + describe "#latest" do + it "should return a single value for 'latest'" do + #gemlist is used for retrieving both local and remote version numbers, and there are cases + # (particularly local) where it makes sense for it to return an array. That doesn't make + # sense for '#latest', though. + provider.class.expects(:gemlist).with({ :justme => 'myresource'}).returns({ + :name => 'myresource', + :ensure => ["3.0"], + :provider => :gem, + }) + provider.latest.should == "3.0" + end + + it "should list from the specified source repository" do + resource[:source] = "http://foo.bar.baz/gems" + provider.class.expects(:gemlist). + with({:justme => 'myresource', :source => "http://foo.bar.baz/gems"}). + returns({ + :name => 'myresource', + :ensure => ["3.0"], + :provider => :gem, + }) + provider.latest.should == "3.0" + end + end + + describe "#instances" do + before do + provider_class.stubs(:command).with(:gemcmd).returns "/my/gem" + end + + it "should return an empty array when no gems installed" do + provider_class.expects(:execute).with(%w{/my/gem list --local}).returns("\n") + provider_class.instances.should == [] + end + + it "should return ensure values as an array of installed versions" do + provider_class.expects(:execute).with(%w{/my/gem list --local}).returns <<-HEREDOC.gsub(/ /, '') + systemu (1.2.0) + vagrant (0.8.7, 0.6.9) + HEREDOC + + provider_class.instances.map {|p| p.properties}.should == [ + {:ensure => ["1.2.0"], :provider => :gem, :name => 'systemu'}, + {:ensure => ["0.8.7", "0.6.9"], :provider => :gem, :name => 'vagrant'} + ] + end + + it "should ignore platform specifications" do + provider_class.expects(:execute).with(%w{/my/gem list --local}).returns <<-HEREDOC.gsub(/ /, '') + systemu (1.2.0) + nokogiri (1.6.1, 1.4.4.1) + HEREDOC + + provider_class.instances.map {|p| p.properties}.should == [ + {:ensure => ["1.2.0"], :provider => :gem, :name => 'systemu'}, + {:ensure => ["1.6.1", "1.4.4.1"], :provider => :gem, :name => 'nokogiri'} + ] + end + + it "should not fail when an unmatched line is returned" do + provider_class.expects(:execute).with(%w{/my/gem list --local}). + returns(File.read(my_fixture('line-with-1.8.5-warning'))) + + provider_class.instances.map {|p| p.properties}. + should == [{:provider=>:gem, :ensure=>["0.3.2"], :name=>"columnize"}, + {:provider=>:gem, :ensure=>["1.1.3"], :name=>"diff-lcs"}, + {:provider=>:gem, :ensure=>["0.0.1"], :name=>"metaclass"}, + {:provider=>:gem, :ensure=>["0.10.5"], :name=>"mocha"}, + {:provider=>:gem, :ensure=>["0.8.7"], :name=>"rake"}, + {:provider=>:gem, :ensure=>["2.9.0"], :name=>"rspec-core"}, + {:provider=>:gem, :ensure=>["2.9.1"], :name=>"rspec-expectations"}, + {:provider=>:gem, :ensure=>["2.9.0"], :name=>"rspec-mocks"}, + {:provider=>:gem, :ensure=>["0.9.0"], :name=>"rubygems-bundler"}, + {:provider=>:gem, :ensure=>["1.11.3.3"], :name=>"rvm"}] + end + end + + #describe "listing gems" do + # describe "searching for a single package" do + # it "searches for an exact match" do + # provider_class.expects(:execute).with(includes('^bundler$')).returns(File.read(my_fixture('gem-list-single-package'))) + # expected = {:name => 'bundler', :ensure => %w[1.6.2], :provider => :fluentgem} + # expect(provider_class.gemlist({:justme => 'bundler'})).to eq(expected) + # end + # end + #end +end diff --git a/fluentd/spec_helper.rb b/fluentd/spec_helper.rb new file mode 100644 index 000000000..3d9200524 --- /dev/null +++ b/fluentd/spec_helper.rb @@ -0,0 +1 @@ +require 'puppetlabs_spec_helper/module_spec_helper' \ No newline at end of file diff --git a/fluentd/templates/filter.erb b/fluentd/templates/filter.erb new file mode 100644 index 000000000..b5b059eef --- /dev/null +++ b/fluentd/templates/filter.erb @@ -0,0 +1,30 @@ +> + type <%= @type %> +<%- if @input_key != '' -%> + input_key <%= @input_key %> +<%- if @regexp != '' -%> + regexp <%= @regexp %> +<%- end -%> +<%- if @exclude != '' -%> + exclude <%= @exclude %> +<%- end -%> +<%- end -%> +<%- @config.sort_by{|key,val|key}.each do |key, val| -%> + <%= key %> <%= val %> +<%- end -%> +<%- if @output_tag != '' -%> + tag <%= @output_tag %> +<%- end -%> +<%- if @add_tag_prefix != '' -%> + add_tag_prefix <%= @add_tag_prefix %> +<%- end -%> +<%- if @remove_tag_prefix != '' -%> + remove_tag_prefix <%= @remove_tag_prefix %> +<%- end -%> +<%- if @add_tag_suffix != '' -%> + add_tag_suffix <%= @add_tag_suffix %> +<%- end -%> +<%- if @remove_tag_suffix != '' -%> + remove_tag_suffix <%= @remove_tag_suffix %> +<%- end -%> + diff --git a/fluentd/templates/forest_match.erb b/fluentd/templates/forest_match.erb new file mode 100644 index 000000000..2b7053b25 --- /dev/null +++ b/fluentd/templates/forest_match.erb @@ -0,0 +1,21 @@ +> + type <%= @type %> +<% @config.sort_by{|key,val|key}.each do |key, val| -%> + <% if val.class == Hash && key == 'template' %> + + <% else -%> + <%= key %> <%= val %> + <% end -%> +<% end -%> +<% @servers.each do |server| -%> + +<% @server.sort_by{|key,val|key}.each do |key, val| -%> + <%= key %> <%= val %> +<% end -%> + +<% end -%> + \ No newline at end of file diff --git a/fluentd/templates/match.erb b/fluentd/templates/match.erb new file mode 100644 index 000000000..1fc09f9c5 --- /dev/null +++ b/fluentd/templates/match.erb @@ -0,0 +1,33 @@ +> + type <%= @type %> + <%- if @type == 'copy' -%> + <%- @config.each do |outcopy| -%> + + <%- outcopy.sort_by{|key,val|key}.each do |key, val| -%> + <%- if key == 'servers' -%> + <%- val.each do |server| -%> + + <%- server.sort_by{|k,v|k}.each do |k,v| -%> + <%= k %> <%= v %> + <%- end -%> + + <%- end -%> + <%- else -%> + <%= key %> <%= val %> + <%- end -%> + <%- end -%> + + <%- end -%> + <%- else -%> + <%- @config.sort_by{|key,val|key}.each do |key, val| -%> + <%= key %> <%= val %> + <%- end -%> + <%- end -%> + <%- @servers.each do |server| -%> + + <%- server.sort_by{|key,val|key}.each do |key, val| -%> + <%= key %> <%= val %> + <%- end -%> + + <%- end -%> + diff --git a/fluentd/templates/source.erb b/fluentd/templates/source.erb new file mode 100644 index 000000000..ea5a2aa88 --- /dev/null +++ b/fluentd/templates/source.erb @@ -0,0 +1,16 @@ + + type <%= @type %> +<% @config.sort_by{|key,val|key}.each do |key, val| -%> + <%= key %> <%= val %> +<% end -%> +<% @pattern.each do |pat| -%> + +<% pat.sort_by{|key,val|key}.each do |key, val| -%> + <%= key %> <%= val %> +<% end -%> + +<% end -%> +<% if @format != false -%> format <%= @format %><% end %> +<% if @time_format != false -%> time_format <%= @time_format %><% end %> +<% if @tag != false -%> tag <%= @tag %><% end %> + diff --git a/fluentd/templates/td-agent.conf.erb b/fluentd/templates/td-agent.conf.erb new file mode 100644 index 000000000..dfced4343 --- /dev/null +++ b/fluentd/templates/td-agent.conf.erb @@ -0,0 +1,2 @@ +# Include config files in the ./config.d directory +<% if @version == '2' -%>@<% end %>include config.d/*.conf From 2c532dc3418f256d5cd4443201bcbe30fdca528b Mon Sep 17 00:00:00 2001 From: Lukas Bezdicka Date: Fri, 11 Dec 2015 16:26:55 +0100 Subject: [PATCH 03/35] Fix typo in Puppetfile made in sensu module Change-Id: I9e995e3a1740e69d284efc957b0e02052be139cb --- Puppetfile | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Puppetfile b/Puppetfile index 002e657d8..1db6e2d81 100644 --- a/Puppetfile +++ b/Puppetfile @@ -215,6 +215,10 @@ mod 'rsync', :git => 'https://github.com/puppetlabs/puppetlabs-rsync.git' mod 'sahara', + :commit => '6862557a4db48b01176e30be6877fda5503f085b', + :git => 'https://github.com/openstack/puppet-sahara.git' + +mod 'sensu', :commit => 'bc5e501e9b6c7ac38181c8749fb6a04b24efdb94', :git => 'https://github.com/sensu/sensu-puppet.git' From cb8cb66a907a430b87b39a7fc48afc298ca064d0 Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Tue, 15 Dec 2015 16:42:05 -0500 Subject: [PATCH 04/35] Sync to liberty. Update apache to cdd81dbf3463c73e6abfaf937f929821baff73dd Change-Id: Ibd412d93d78d5cb85affdcd4dbab314ab945ff54 Update ceph to e4b05caf4825af51f679f0618229dce4c3388a5f Change-Id: I31c3b404700ac15bd469184bcecf526a42649cda Update elasticsearch to 171a4a4dc89ad4543f486ed90040b5fbea8d81f0 Change-Id: I9afcf819d18363e960f5d9be6c10395d5889fb85 Update kibana3 to 6ca9631fbe82766134f98e2e8780bb91e7cd3f0e Change-Id: I476afa1a0e82a239e86af5536374145eb461c547 Update redis to 62c0c71e59182ebc252a9405db68b7a1538a745e Change-Id: I2bcecd363d6346b06eacad56a65477a5c08e9bca Update rabbitmq to 3d74c2d77bd482f59ea919e54d24589487221702 Change-Id: Icfb4d5b63c758d65140ef9aa6a92d18bee1084a4 Update sensu to 553994fb8724f84ec820a0a36b347a148d4495e3 Change-Id: Icdf24bbe564175afca8a45e1dd6821c0617e359f Update uchiwa to b3f9ed80d33f6f4f6ca69b60fc3c5aa74fb3ac85 Change-Id: I571fc4cee7f8ab9711351686cb3edb3955297d19 Update ceilometer to b9434a8d9d35300fb8247b862823bf6acbde15aa b9434a8d9d35300fb8247b862823bf6acbde15aa Merge "Implement Gnocchi dispatcher" into stable/liberty e934cdf0b13b2db5fbdde2a271c9aaf9d01b1330 Implement Gnocchi dispatcher Change-Id: I64591b9880712df7b11345cd6866a9c289fd4bb4 Update contrail to 83471677d5b7b7a1e26c7ccb20f5ab355f41efae Change-Id: I2b1b3b634ba2983235474c7b44f9338ed8de1d9d Update midonet to a4bb534be34a806811df51874a5bba132ca24724 Change-Id: I15bcd9c263cfbcabd5295bb854029995016bdcf5 Update opendaylight to 33353e8d869f4956d706fedd6f4f9c8b4b59a4e0 Change-Id: Id4bb60f4b5805e2b698553cba8f6cd76b93a269e Update sahara to 811a5e9e95ade9a3d8c741e8cbfdc4e57eaca503 811a5e9e95ade9a3d8c741e8cbfdc4e57eaca503 Add deprecation warning for Qpid rpc driver 1cd44f1305d8db430c5ead1384ff7bf9eca63fcd release: prepare 7.0.0 (liberty) Change-Id: I760474ab8b053a7758a1ff78c6a0a47e2e8e4c4e Update tripleo to aaef6b770e4af987400bb8c674304e5b47b30982 aaef6b770e4af987400bb8c674304e5b47b30982 Set defaultbranch in .gitreview to stable/liberty Change-Id: Id9bb57e68e99c074c22778e4820efbb364132315 Update openstacklib to 142f9d1be907c45115ea4d92277278a682fb53f7 142f9d1be907c45115ea4d92277278a682fb53f7 Fix unit tests failing against Puppet 4.3.0 7bca6e2146631027cd391c1b86e6bb887893c69b Avoid duplicate client resource Change-Id: I48a282f3e1191aa43b3f0e54e2fab3a37e82080e --- Puppetfile | 30 +- apache/CHANGELOG.md | 16 + apache/README.md | 63 +++- apache/manifests/init.pp | 5 +- apache/manifests/mod/itk.pp | 11 + apache/manifests/mod/ldap.pp | 7 +- apache/manifests/mpm.pp | 38 +- apache/manifests/vhost.pp | 41 +- apache/manifests/vhost/custom.pp | 37 ++ apache/metadata.json | 4 +- .../spec/acceptance/apache_parameters_spec.rb | 16 +- apache/spec/acceptance/apache_ssl_spec.rb | 2 +- apache/spec/acceptance/class_spec.rb | 2 +- apache/spec/acceptance/custom_config_spec.rb | 2 +- apache/spec/acceptance/default_mods_spec.rb | 2 +- apache/spec/acceptance/itk_spec.rb | 35 +- apache/spec/acceptance/mod_fcgid_spec.rb | 4 +- apache/spec/acceptance/mod_mime_spec.rb | 2 +- .../spec/acceptance/mod_negotiation_spec.rb | 2 +- apache/spec/acceptance/mod_pagespeed_spec.rb | 2 +- apache/spec/acceptance/mod_passenger_spec.rb | 2 +- apache/spec/acceptance/mod_php_spec.rb | 2 +- apache/spec/acceptance/mod_proxy_html_spec.rb | 2 +- apache/spec/acceptance/mod_security_spec.rb | 2 +- apache/spec/acceptance/mod_suphp_spec.rb | 56 ++- apache/spec/acceptance/prefork_worker_spec.rb | 4 +- apache/spec/acceptance/service_spec.rb | 2 +- apache/spec/acceptance/unsupported_spec.rb | 13 - apache/spec/acceptance/vhost_spec.rb | 121 +++--- apache/spec/classes/dev_spec.rb | 1 - apache/spec/classes/mod/itk_spec.rb | 8 + apache/spec/classes/mod/ldap_spec.rb | 78 ++++ apache/spec/classes/mod/pagespeed_spec.rb | 11 +- apache/spec/classes/mod/passenger_spec.rb | 1 - apache/spec/classes/mod/php_spec.rb | 2 +- apache/spec/defines/vhost_custom_spec.rb | 99 +++++ apache/spec/defines/vhost_spec.rb | 28 +- apache/spec/spec_helper_acceptance.rb | 36 +- apache/templates/httpd.conf.erb | 1 + apache/templates/mod/fastcgi.conf.erb | 4 +- apache/templates/mod/ldap.conf.erb | 3 + apache/templates/mod/nss.conf.erb | 2 +- apache/templates/mod/pagespeed.conf.erb | 8 +- apache/templates/mod/php5.conf.erb | 5 +- apache/templates/vhost/_directories.erb | 4 +- apache/templates/vhost/_limits.erb | 5 - apache/templates/vhost/_ssl.erb | 6 - apache/templates/vhost/_sslproxy.erb | 14 + .../examples/ceilometer_with_gnocchi.pp | 34 ++ ceilometer/manifests/collector.pp | 28 +- ceilometer/manifests/dispatcher/gnocchi.pp | 43 +++ .../acceptance/ceilometer_with_mysql_spec.rb | 1 + .../acceptance/ceilometer_wsgi_apache_spec.rb | 1 + .../spec/classes/ceilometer_collector_spec.rb | 2 + .../ceilometer_dispatcher_gnocchi_spec.rb | 54 +++ ceph/.gitreview | 2 +- ceph/Puppetfile | 2 +- ceph/README.md | 6 +- ceph/Rakefile | 1 - ceph/examples/common.yaml | 4 +- ceph/examples/nodes/first.yaml | 2 +- ceph/examples/nodes/second.yaml | 2 +- ceph/manifests/repo.pp | 2 +- ceph/metadata.json | 6 +- ceph/spec/acceptance/ceph_rgw_spec.rb | 33 +- ceph/spec/classes/ceph_osds_spec.rb | 4 +- ceph/spec/classes/ceph_profile_osd_spec.rb | 6 +- ceph/spec/classes/ceph_repo_spec.rb | 8 +- ceph/spec/defines/ceph_osd_spec.rb | 70 ++-- ceph/spec/defines/ceph_rgw_apache_spec.rb | 15 +- ceph/spec/defines/ceph_rgw_keystone_spec.rb | 15 +- ceph/spec/fixtures/hieradata/common.yaml | 4 +- ceph/spec/fixtures/hieradata/nodes/first.yaml | 2 +- ceph/spec/spec_helper_acceptance.rb | 2 +- ceph/spec/spec_helper_system.rb | 2 +- ceph/spec/system/ceph_osd_spec.rb | 2 +- ceph/spec/system/ceph_profile_osd_spec.rb | 2 +- .../provider/ceph_config/ini_setting_spec.rb | 2 +- contrail/README.md | 2 +- contrail/manifests/control/config.pp | 10 + contrail/metadata.json | 6 +- contrail/templates/contrail-named.conf.erb | 2 +- elasticsearch/.fixtures.yml | 2 +- elasticsearch/.pmtignore | 4 + elasticsearch/CHANGELOG.md | 27 ++ elasticsearch/Gemfile | 2 +- elasticsearch/README.md | 11 +- .../provider/elasticsearch_plugin/plugin.rb | 130 +++++++ .../lib/puppet/type/elasticsearch_plugin.rb | 31 ++ elasticsearch/manifests/config.pp | 14 +- elasticsearch/manifests/init.pp | 19 +- elasticsearch/manifests/instance.pp | 1 + elasticsearch/manifests/package.pp | 45 +-- elasticsearch/manifests/plugin.pp | 63 +--- elasticsearch/manifests/repo.pp | 39 +- elasticsearch/metadata.json | 2 +- .../spec/acceptance/004_plugin_spec.rb | 45 +++ ..._datapath_spec.rb => 009_datapath_spec.rb} | 0 .../spec/acceptance/021_es2x_spec.rb | 351 ++++++++++++++++++ elasticsearch/spec/acceptance/xplugins001.rb | 91 +++++ .../classes/000_elasticsearch_init_spec.rb | 5 +- elasticsearch/spec/classes/001_hiera_spec.rb | 5 +- .../classes/005_elasticsearch_repo_spec.rb | 54 ++- .../defines/004_elasticsearch_plugin_spec.rb | 107 +----- elasticsearch/spec/spec_acceptance_common.rb | 1 + elasticsearch/spec/spec_helper_acceptance.rb | 8 +- .../spec/unit/provider/plugin_spec.rb | 112 ++++++ elasticsearch/spec/unit/type/plugin_spec.rb | 57 +++ .../etc/init.d/elasticsearch.Debian.erb | 1 + .../etc/init.d/elasticsearch.RedHat.erb | 1 + midonet/.gitreview | 1 - midonet/README.md | 12 +- midonet/metadata.json | 5 +- midonet/test/init.pp | 19 - .../default/bats/verify_server.bats | 160 -------- opendaylight/CONTRIBUTING.markdown | 3 +- opendaylight/Rakefile | 21 +- opendaylight/Vagrantfile | 47 +++ .../nodesets/ubuntu-1404-docker.yml | 7 + .../spec/acceptance/nodesets/ubuntu-1504.yml | 12 - opendaylight/spec/spec_helper_acceptance.rb | 2 +- openstacklib/manifests/openstackclient.pp | 5 +- .../openstacklib_db_postgresql_spec.rb | 29 +- .../provider/rabbitmq_erlang_cookie/ruby.rb | 9 +- rabbitmq/manifests/init.pp | 4 +- rabbitmq/manifests/params.pp | 2 +- rabbitmq/spec/classes/rabbitmq_spec.rb | 66 +++- redis/.gitignore | 6 +- redis/Gemfile | 10 + redis/README.md | 10 + redis/manifests/config.pp | 61 +-- redis/manifests/init.pp | 46 ++- redis/manifests/params.pp | 15 +- redis/manifests/preinstall.pp | 10 +- redis/manifests/sentinel.pp | 15 +- redis/metadata.json | 2 +- redis/spec/acceptance/nodesets/default.yml | 11 + redis/spec/acceptance/redis_spec.rb | 23 ++ redis/spec/classes/redis_spec.rb | 81 ++++ redis/spec/spec_helper_acceptance.rb | 31 ++ redis/templates/redis.conf.erb | 27 +- sahara/CHANGELOG.md | 33 ++ sahara/README.md | 2 +- sahara/Rakefile | 2 +- sahara/manifests/init.pp | 2 + sahara/manifests/notify/qpid.pp | 1 + sahara/metadata.json | 6 +- sahara/spec/spec_helper_acceptance.rb | 2 +- sensu/README.md | 2 +- .../provider/sensu_client_config/json.rb | 2 +- .../sensu_enterprise_dashboard_config/json.rb | 8 +- sensu/lib/puppet/type/sensu_client_config.rb | 16 +- sensu/manifests/repo/apt.pp | 7 + sensu/tests/provision_client.sh | 4 +- sensu/tests/provision_server.sh | 4 +- tripleo/.fixtures.yml | 21 -- tripleo/.gitreview | 1 + .../lib/puppet/parser/functions/extract_id.rb | 14 - .../functions/list_to_zookeeper_hash.rb | 24 -- tripleo/manifests/cluster/cassandra.pp | 69 ---- tripleo/manifests/cluster/zookeeper.pp | 65 ---- tripleo/manifests/loadbalancer.pp | 1 - tripleo/manifests/network/midonet/agent.pp | 61 --- tripleo/manifests/network/midonet/api.pp | 117 ------ .../classes/tripleo_cluster_cassandra_spec.rb | 45 --- .../classes/tripleo_cluster_zookeeper_spec.rb | 115 ------ tripleo/spec/classes/tripleo_firewall_spec.rb | 5 +- .../classes/tripleo_midonet_agent_spec.rb | 62 ---- .../spec/classes/tripleo_midonet_api_spec.rb | 72 ---- tripleo/spec/spec_helper.rb | 4 - uchiwa/.gitignore | 1 + uchiwa/CHANGELOG.md | 117 ++++++ uchiwa/Gemfile | 6 +- uchiwa/Gemfile.lock | 77 ++++ uchiwa/Rakefile | 9 +- uchiwa/manifests/init.pp | 36 +- uchiwa/manifests/params.pp | 6 +- uchiwa/manifests/repo/apt.pp | 2 +- uchiwa/metadata.json | 11 +- uchiwa/spec/classes/uchiwa_spec.rb | 22 +- uchiwa/templates/etc/sensu/uchiwa.json.erb | 21 +- 181 files changed, 2736 insertions(+), 1537 deletions(-) create mode 100644 apache/manifests/vhost/custom.pp delete mode 100644 apache/spec/acceptance/unsupported_spec.rb create mode 100644 apache/spec/classes/mod/ldap_spec.rb create mode 100644 apache/spec/defines/vhost_custom_spec.rb delete mode 100644 apache/templates/vhost/_limits.erb create mode 100644 apache/templates/vhost/_sslproxy.erb create mode 100644 ceilometer/examples/ceilometer_with_gnocchi.pp create mode 100644 ceilometer/manifests/dispatcher/gnocchi.pp create mode 100644 ceilometer/spec/classes/ceilometer_dispatcher_gnocchi_spec.rb create mode 100644 elasticsearch/lib/puppet/provider/elasticsearch_plugin/plugin.rb create mode 100644 elasticsearch/lib/puppet/type/elasticsearch_plugin.rb rename elasticsearch/spec/acceptance/{005_datapath_spec.rb => 009_datapath_spec.rb} (100%) create mode 100644 elasticsearch/spec/acceptance/021_es2x_spec.rb create mode 100644 elasticsearch/spec/acceptance/xplugins001.rb create mode 100644 elasticsearch/spec/unit/provider/plugin_spec.rb create mode 100644 elasticsearch/spec/unit/type/plugin_spec.rb delete mode 100644 midonet/test/init.pp delete mode 100644 midonet/test/integration/default/bats/verify_server.bats create mode 100644 opendaylight/Vagrantfile create mode 100644 opendaylight/spec/acceptance/nodesets/ubuntu-1404-docker.yml delete mode 100644 opendaylight/spec/acceptance/nodesets/ubuntu-1504.yml create mode 100644 redis/spec/acceptance/nodesets/default.yml create mode 100644 redis/spec/acceptance/redis_spec.rb create mode 100644 redis/spec/spec_helper_acceptance.rb delete mode 100644 tripleo/lib/puppet/parser/functions/extract_id.rb delete mode 100644 tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb delete mode 100644 tripleo/manifests/cluster/cassandra.pp delete mode 100644 tripleo/manifests/cluster/zookeeper.pp delete mode 100644 tripleo/manifests/network/midonet/agent.pp delete mode 100644 tripleo/manifests/network/midonet/api.pp delete mode 100644 tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb delete mode 100644 tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb delete mode 100644 tripleo/spec/classes/tripleo_midonet_agent_spec.rb delete mode 100644 tripleo/spec/classes/tripleo_midonet_api_spec.rb create mode 100644 uchiwa/CHANGELOG.md diff --git a/Puppetfile b/Puppetfile index 1db6e2d81..c56006d52 100644 --- a/Puppetfile +++ b/Puppetfile @@ -3,7 +3,7 @@ mod 'aodh', :git => 'https://github.com/openstack/puppet-aodh.git' mod 'apache', - :commit => '13b79e2446ca1ee08301c03bc2ee2579114da84a', + :commit => 'cdd81dbf3463c73e6abfaf937f929821baff73dd', :git => 'https://github.com/puppetlabs/puppetlabs-apache.git' mod 'aviator', @@ -15,11 +15,11 @@ mod 'cassandra', :git => 'https://github.com/locp/cassandra.git' mod 'ceilometer', - :commit => 'd303048dd29f8e645a992aa8ec23799bcc8eedfb', + :commit => 'b9434a8d9d35300fb8247b862823bf6acbde15aa', :git => 'https://github.com/openstack/puppet-ceilometer.git' mod 'ceph', - :commit => 'c60c6209ac799555b94bf62ce1c2cf00084e42d2', + :commit => 'e4b05caf4825af51f679f0618229dce4c3388a5f', :git => 'https://github.com/stackforge/puppet-ceph.git' mod 'certmonger', @@ -39,7 +39,7 @@ mod 'concat', :git => 'https://github.com/puppetlabs/puppetlabs-concat.git' mod 'contrail', - :commit => '63803ba710ade71ce93c6e5c94054f193d66d56e', + :commit => '83471677d5b7b7a1e26c7ccb20f5ab355f41efae', :git => 'https://github.com/redhat-cip/puppet-contrail.git' mod 'corosync', @@ -51,7 +51,7 @@ mod 'datacat', :git => 'http://github.com/richardc/puppet-datacat' mod 'elasticsearch', - :commit => 'b930ab8', + :commit => '171a4a4dc89ad4543f486ed90040b5fbea8d81f0', :git => 'https://github.com/elastic/puppet-elasticsearch.git' mod 'firewall', @@ -119,7 +119,7 @@ mod 'keystone', :git => 'https://github.com/openstack/puppet-keystone.git' mod 'kibana3', - :commit => '6ca9631', + :commit => '6ca9631fbe82766134f98e2e8780bb91e7cd3f0e', :git => 'https://github.com/thejandroman/puppet-kibana3.git' mod 'manila', @@ -131,7 +131,7 @@ mod 'memcached', :git => 'https://github.com/saz/puppet-memcached.git' mod 'midonet', - :commit => 'e24515c135a03096fc69651dfadac5b545c74538', + :commit => 'a4bb534be34a806811df51874a5bba132ca24724', :git => 'https://github.com/midonet/puppet-midonet.git' mod 'module-collectd', @@ -175,7 +175,7 @@ mod 'ntp', :git => 'https://github.com/puppetlabs/puppetlabs-ntp' mod 'opendaylight', - :commit => '3aa0ccf', + :commit => '33353e8d869f4956d706fedd6f4f9c8b4b59a4e0', :git => 'https://github.com/dfarrell07/puppet-opendaylight.git' mod 'openstack_extras', @@ -183,7 +183,7 @@ mod 'openstack_extras', :git => 'https://github.com/openstack/puppet-openstack_extras.git' mod 'openstacklib', - :commit => '47a0c6392fad4c1fb11811b17968576033a24baa', + :commit => '142f9d1be907c45115ea4d92277278a682fb53f7', :git => 'https://github.com/openstack/puppet-openstacklib.git' mod 'pacemaker', @@ -199,11 +199,11 @@ mod 'qpid', :git => 'https://github.com/dprince/puppet-qpid' mod 'rabbitmq', - :commit => '151391f03b1d9dcaf895123aa98b0a0d67e30540', + :commit => '3d74c2d77bd482f59ea919e54d24589487221702', :git => 'https://github.com/puppetlabs/puppetlabs-rabbitmq.git' mod 'redis', - :commit => '47b6fe8a64dce5447105ab59aa1fa2984d6ef1d0', + :commit => '62c0c71e59182ebc252a9405db68b7a1538a745e', :git => 'https://github.com/arioch/puppet-redis.git' mod 'remote', @@ -215,11 +215,11 @@ mod 'rsync', :git => 'https://github.com/puppetlabs/puppetlabs-rsync.git' mod 'sahara', - :commit => '6862557a4db48b01176e30be6877fda5503f085b', + :commit => '811a5e9e95ade9a3d8c741e8cbfdc4e57eaca503', :git => 'https://github.com/openstack/puppet-sahara.git' mod 'sensu', - :commit => 'bc5e501e9b6c7ac38181c8749fb6a04b24efdb94', + :commit => '553994fb8724f84ec820a0a36b347a148d4495e3', :git => 'https://github.com/sensu/sensu-puppet.git' mod 'snmp', @@ -259,7 +259,7 @@ mod 'tomcat', :git => 'https://github.com/puppetlabs/puppetlabs-tomcat.git' mod 'tripleo', - :commit => '74b589497c9efe400403f2d446aa6335abdb89d5', + :commit => 'aaef6b770e4af987400bb8c674304e5b47b30982', :git => 'https://github.com/openstack/puppet-tripleo.git' mod 'trove', @@ -271,7 +271,7 @@ mod 'tuskar', :git => 'https://github.com/openstack/puppet-tuskar.git' mod 'uchiwa', - :commit => 'e3dc4fc3d5291d08a23f16843c673932adf450d5', + :commit => 'b3f9ed80d33f6f4f6ca69b60fc3c5aa74fb3ac85', :git => 'https://github.com/Yelp/puppet-uchiwa.git' mod 'vcsrepo', diff --git a/apache/CHANGELOG.md b/apache/CHANGELOG.md index fb33a1363..db27b34ba 100644 --- a/apache/CHANGELOG.md +++ b/apache/CHANGELOG.md @@ -1,3 +1,19 @@ +## UNRELEASED +### Summary + +TODO + +### + +#### Security + +* apache::mod::php now uses FilesMatch to configure the php handler. This is following the recommended upstream configuration guidelines (http://php.net/manual/en/install.unix.apache2.php#example-20) and distribution's default config (e.g.: http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/vivid/php5/vivid/view/head:/debian/php5.conf). It avoids inadvertently exposing the PHP handler to executing uploads with names like 'file.php.jpg', but might impact setups with unusual requirements. + +## Supported Release 1.7.1 +###Summary + +Small release for support of newer PE versions. This increments the version of PE in the metadata.json file. + ## 2015-11-17 - Supported Release 1.7.0 ### Summary This release includes many new features and bugfixes. There are test, documentation and misc improvements. diff --git a/apache/README.md b/apache/README.md index a8c4b47df..57ba525bf 100644 --- a/apache/README.md +++ b/apache/README.md @@ -51,6 +51,7 @@ [`apache::mod::ext_filter`]: #class-apachemodext_filter [`apache::mod::geoip`]: #class-apachemodgeoip [`apache::mod::itk`]: #class-apachemoditk +[`apache::mod::ldap`]: #class-apachemodldap [`apache::mod::passenger`]: #class-apachemodpassenger [`apache::mod::peruser`]: #class-apachemodperuser [`apache::mod::prefork`]: #class-apachemodprefork @@ -64,6 +65,7 @@ [`apache::params`]: #class-apacheparams [`apache::version`]: #class-apacheversion [`apache::vhost`]: #define-apachevhost +[`apache::vhost::custom`]: #define-apachevhostcustom [`apache::vhost::WSGIImportScript`]: #wsgiimportscript [Apache HTTP Server]: http://httpd.apache.org [Apache modules]: http://httpd.apache.org/docs/current/mod/ @@ -123,6 +125,7 @@ [`KeepAliveTimeout`]: http://httpd.apache.org/docs/current/mod/core.html#keepalivetimeout [`keepalive` parameter]: #keepalive [`keepalive_timeout`]: #keepalive_timeout +[`limitreqfieldsize`]: https://httpd.apache.org/docs/current/mod/core.html#limitrequestfieldsize [`lib`]: #lib [`lib_path`]: #lib_path @@ -154,6 +157,7 @@ [`mod_fcgid`]: https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html [`mod_geoip`]: http://dev.maxmind.com/geoip/legacy/mod_geoip2/ [`mod_info`]: https://httpd.apache.org/docs/current/mod/mod_info.html +[`mod_ldap`]: https://httpd.apache.org/docs/2.2/mod/mod_ldap.html [`mod_mpm_event`]: https://httpd.apache.org/docs/current/mod/event.html [`mod_negotiation`]: http://httpd.apache.org/docs/current/mod/mod_negotiation.html [`mod_pagespeed`]: https://developers.google.com/speed/pagespeed/module/?hl=en @@ -740,6 +744,7 @@ apache::balancer { 'puppet01': - [Define: apache::mod](#define-apachemod) - [Define: apache::namevirtualhost](#define-apachenamevirtualhost) - [Define: apache::vhost](#define-apachevhost) + - [Define: apache::vhost::custom](#define-apachevhostcustom) - [**Private Defines**](#private-defines) - [Define: apache::default_mods::load](#define-default_mods-load) - [Define: apache::peruser::multiplexer](#define-apacheperusermultiplexer) @@ -1308,7 +1313,7 @@ Installs and manages [`mod_alias`][]. #### Class: `apache::mod::disk_cache` -Installs and configures [`mod_disk_cache`][] on Apache 2.2, or [`mod_cache_disk`][] on Apache 2.4. The default cache root depends on the Apache version and operating system: +Installs and configures [`mod_disk_cache`][] on Apache 2.2, or [`mod_cache_disk`][] on Apache 2.4. The default cache root depends on the Apache version and operating system: - **Debian**: `/var/cache/apache2/mod_cache_disk` - **FreeBSD**: `/var/cache/mod_cache_disk` @@ -1476,6 +1481,23 @@ Installs and manages [`mod_info`][], which provides a comprehensive overview of - `apache_version`: Default: `$::apache::apache_version`, - `restrict_access`: Determines whether to enable access restrictions. If 'false', the `allow_from` whitelist is ignored and any IP address can access `/server-info`. Valid options: Boolean. Default: 'true'. +##### Class: `apache::mod::ldap` + +Installs and configures [`mod_ldap`][]. Allows you to modify the +[`LDAPTrustedGlobalCert`](https://httpd.apache.org/docs/2.2/mod/mod_ldap.html#ldaptrustedglobalcert) Directive: + +~~~puppet +class { 'apache::mod::ldap': + ldap_trusted_global_cert_file => '/etc/pki/tls/certs/ldap-trust.crt' + ldap_trusted_global_cert_type => 'CA_DER', +} +~~~ + +**Parameters within `apache::mod::ldap`:** + +- `ldap_trusted_global_cert_file`: Path and file name of the trusted CA certificates to use when establishing SSL or TLS connections to an LDAP server. +- `ldap_trusted_global_cert_type`: The global trust certificate format. Defaults to 'CA_BASE64'. + ##### Class: `apache::mod::negotiation` Installs and configures [`mod_negotiation`][]. @@ -1491,7 +1513,7 @@ Installs and manages [`mod_pagespeed`], a Google module that rewrites web pages While this Apache module requires the `mod-pagespeed-stable` package, Puppet **doesn't** manage the software repositories required to automatically install the package. If you declare this class when the package is either not installed or not available to your package manager, your Puppet run will fail. -**Parameters within `apache::mod::info`**: +**Parameters within `apache::mod::pagespeed`**: - `inherit_vhost_config`: Default: 'on'. - `filter_xhtml`: Default: false. @@ -1525,7 +1547,7 @@ While this Apache module requires the `mod-pagespeed-stable` package, Puppet **d - `allow_pagespeed_console`: Default: []. - `allow_pagespeed_message`: Default: []. - `message_buffer_size`: Default: 100000. -- `additional_configuration`: Default: { }. +- `additional_configuration`: Default: { }. A hash of directive/value pairs or an array of lines to insert at the end of the pagespeed configuration. The class's parameters correspond to the module's directives. See the [module's documentation][`mod_pagespeed`] for details. @@ -2084,10 +2106,6 @@ Specifies the service name that will be used by Apache for authentication. Corre This option enables credential saving functionality. Default is 'off' -##### `limit_request_field_size` - -[Limits](http://httpd.apache.org/docs/2.4/mod/core.html#limitrequestfieldsize) the size of the HTTP request header allowed from the client. Default is 'undef'. - ##### `logroot` Specifies the location of the virtual host's logfiles. Defaults to '/var/log//'. @@ -3167,6 +3185,15 @@ Sets the [SSLProxyMachineCertificateFile](http://httpd.apache.org/docs/current/m } ~~~ +##### `ssl_proxy_check_peer_cn` + +Sets the [SSLProxyMachinePeerCN](http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxycheckpeercn) directive, which specified whether the remote server certificate's CN field is compared against the hostname of the request URL . Defaults to 'undef'. + + +##### `ssl_proxy_check_peer_name` + +Sets the [SSLProxyMachinePeerName](http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxycheckpeername) directive, which specified whether the remote server certificate's CN field is compared against the hostname of the request URL . Defaults to 'undef'. + ##### `ssl_options` Sets the [SSLOptions](http://httpd.apache.org/docs/current/mod/mod_ssl.html#ssloptions) directive, which configures various SSL engine run-time options. This is the global setting for the given vhost and can be a string or an array. Defaults to 'undef'. @@ -3201,6 +3228,8 @@ Specifies whether or not to use [SSLProxyEngine](http://httpd.apache.org/docs/cu This type is intended for use with mod_fastcgi. It allows you to define one or more external FastCGI servers to handle specific file types. +** Note ** If using Ubuntu 10.04+, you'll need to manually enable the multiverse repository. + Ex: ~~~ puppet @@ -3248,6 +3277,26 @@ A unique alias. This is used internally to link the action with the FastCGI serv The MIME-type of the file to be processed by the FastCGI server. +#### Define: `apache::vhost::custom` + +The `apache::vhost::custom` is a thin wrapper to the `apache::custom_config`` +define. We are simply overriding some of the default settings specifc to the +vhost directory in Apache. + +**Parameters within `apache::vhost::custom`**: + +##### `content` + +Sets the configuration file's content. + +##### `ensure` + +Specifies if the vhost file is present or absent. Defaults to 'present'. + +##### `priority` + +Sets the relative load-order for Apache HTTPD VirtualHost configuration files. Defaults to '25'. + ### Private Defines #### Define: `apache::peruser::multiplexer` diff --git a/apache/manifests/init.pp b/apache/manifests/init.pp index 5a865b761..bb50d0b2e 100644 --- a/apache/manifests/init.pp +++ b/apache/manifests/init.pp @@ -63,6 +63,7 @@ $keepalive = $::apache::params::keepalive, $keepalive_timeout = $::apache::params::keepalive_timeout, $max_keepalive_requests = $::apache::params::max_keepalive_requests, + $limitreqfieldsize = '8190', $logroot = $::apache::params::logroot, $logroot_mode = $::apache::params::logroot_mode, $log_level = $::apache::params::log_level, @@ -92,7 +93,7 @@ default => '(event|itk|prefork|worker)' } - if $mpm_module { + if $mpm_module and $mpm_module != 'false' { # lint:ignore:quoted_booleans validate_re($mpm_module, $valid_mpms_re) } @@ -346,7 +347,7 @@ class { '::apache::default_confd_files': all => $default_confd_files } - if $mpm_module { + if $mpm_module and $mpm_module != 'false' { # lint:ignore:quoted_booleans class { "::apache::mod::${mpm_module}": } } diff --git a/apache/manifests/mod/itk.pp b/apache/manifests/mod/itk.pp index 2be7d832d..990f853ea 100644 --- a/apache/manifests/mod/itk.pp +++ b/apache/manifests/mod/itk.pp @@ -17,6 +17,17 @@ if defined(Class['apache::mod::prefork']) { fail('May not include both apache::mod::itk and apache::mod::prefork on the same node') } + } else { + # prefork is a requirement for itk in 2.4; except on FreeBSD and Gentoo, which are special + if $::osfamily =~ /^(FreeBSD|Gentoo)/ { + if defined(Class['apache::mod::prefork']) { + fail('May not include both apache::mod::itk and apache::mod::prefork on the same node') + } + } else { + if ! defined(Class['apache::mod::prefork']) { + fail('apache::mod::prefork is a prerequisite for apache::mod::itk, please arrange for it to be included.') + } + } } if defined(Class['apache::mod::worker']) { fail('May not include both apache::mod::itk and apache::mod::worker on the same node') diff --git a/apache/manifests/mod/ldap.pp b/apache/manifests/mod/ldap.pp index fbd56d539..d08418671 100644 --- a/apache/manifests/mod/ldap.pp +++ b/apache/manifests/mod/ldap.pp @@ -1,6 +1,11 @@ class apache::mod::ldap ( - $apache_version = $::apache::apache_version, + $apache_version = $::apache::apache_version, + $ldap_trusted_global_cert_file = undef, + $ldap_trusted_global_cert_type = 'CA_BASE64', ){ + if ($ldap_trusted_global_cert_file) { + validate_string($ldap_trusted_global_cert_type) + } ::apache::mod { 'ldap': } # Template uses $apache_version file { 'ldap.conf': diff --git a/apache/manifests/mpm.pp b/apache/manifests/mpm.pp index 9e7734945..153540910 100644 --- a/apache/manifests/mpm.pp +++ b/apache/manifests/mpm.pp @@ -47,17 +47,34 @@ } if $mpm == 'itk' { - file { "${lib_path}/mod_mpm_itk.so": - ensure => link, - target => "${lib_path}/mpm_itk.so" - } + file { "${lib_path}/mod_mpm_itk.so": + ensure => link, + target => "${lib_path}/mpm_itk.so", + require => Package['httpd'], + before => Class['apache::service'], + } } } - if versioncmp($apache_version, '2.4') < 0 { + if $mpm == 'itk' and $::operatingsystem == 'Ubuntu' and $::operatingsystemrelease == '14.04' { + # workaround https://bugs.launchpad.net/ubuntu/+source/mpm-itk/+bug/1286882 + exec { + '/usr/sbin/a2dismod mpm_event': + onlyif => '/usr/bin/test -e /etc/apache2/mods-enabled/mpm_event.load', + require => Package['httpd'], + before => Package['apache2-mpm-itk'], + } + } + + if versioncmp($apache_version, '2.4') < 0 or $mpm == 'itk' { package { "apache2-mpm-${mpm}": ensure => present, } + if $::apache::mod_enable_dir { + Package["apache2-mpm-${mpm}"] { + before => File[$::apache::mod_enable_dir], + } + } } } 'freebsd': { @@ -65,6 +82,9 @@ mpm_module => $mpm } } + 'gentoo': { + # so we don't fail + } 'redhat': { # so we don't fail } @@ -87,10 +107,10 @@ } if $mpm == 'itk' { - file { "${lib_path}/mod_mpm_itk.so": - ensure => link, - target => "${lib_path}/mpm_itk.so" - } + file { "${lib_path}/mod_mpm_itk.so": + ensure => link, + target => "${lib_path}/mpm_itk.so" + } } } diff --git a/apache/manifests/vhost.pp b/apache/manifests/vhost.pp index e2725b8ae..7e48317a8 100644 --- a/apache/manifests/vhost.pp +++ b/apache/manifests/vhost.pp @@ -25,6 +25,8 @@ $ssl_honorcipherorder = undef, $ssl_verify_client = undef, $ssl_verify_depth = undef, + $ssl_proxy_check_peer_cn = undef, + $ssl_proxy_check_peer_name = undef, $ssl_proxy_machine_cert = undef, $ssl_options = undef, $ssl_openssl_conf_cmd = undef, @@ -134,7 +136,6 @@ $krb_verify_kdc = 'on', $krb_servicename = 'HTTP', $krb_save_credentials = 'off', - $limit_request_field_size = undef, ) { # The base class must be included first because it is used by parameter defaults if ! defined(Class['apache']) { @@ -227,15 +228,19 @@ validate_bool($auth_kerb) - if $limit_request_field_size { - validate_integer($limit_request_field_size) - } - # Validate the docroot as a string if: # - $manage_docroot is true if $manage_docroot { validate_string($docroot) } + + if $ssl_proxy_check_peer_cn { + validate_re($ssl_proxy_check_peer_cn,'(^on$|^off$)',"${ssl_proxy_check_peer_cn} is not permitted for ssl_proxy_check_peer_cn. Allowed values are 'on' or 'off'.") + } + if $ssl_proxy_check_peer_name { + validate_re($ssl_proxy_check_peer_name,'(^on$|^off$)',"${ssl_proxy_check_peer_name} is not permitted for ssl_proxy_check_peer_name. Allowed values are 'on' or 'off'.") + } + # Input validation ends if $ssl and $ensure == 'present' { @@ -497,6 +502,8 @@ require => Package['httpd'], notify => Class['apache::service'], } + # NOTE(pabelanger): This code is duplicated in ::apache::vhost::custom and + # needs to be converted into something generic. if $::apache::vhost_enable_dir { $vhost_enable_dir = $::apache::vhost_enable_dir $vhost_symlink_ensure = $ensure ? { @@ -778,13 +785,11 @@ # - $ssl_crl_path # - $ssl_crl # - $ssl_crl_check - # - $ssl_proxyengine # - $ssl_protocol # - $ssl_cipher # - $ssl_honorcipherorder # - $ssl_verify_client # - $ssl_verify_depth - # - $ssl_proxy_machine_cert # - $ssl_options # - $ssl_openssl_conf_cmd # - $apache_version @@ -796,6 +801,19 @@ } } + # Template uses: + # - $ssl_proxyengine + # - $ssl_proxy_check_peer_cn + # - $ssl_proxy_check_peer_name + # - $ssl_proxy_machine_cert + if $ssl_proxyengine { + concat::fragment { "${name}-sslproxy": + target => "${priority_real}${filename}.conf", + order => 210, + content => template('apache/vhost/_sslproxy.erb'), + } + } + # Template uses: # - $auth_kerb # - $krb_method_negotiate @@ -963,15 +981,6 @@ content => template('apache/vhost/_filters.erb'), } } - # Template uses: - # - $limit_request_field_size - if $limit_request_field_size { - concat::fragment { "${name}-limits": - target => "${priority_real}${filename}.conf", - order => 330, - content => template('apache/vhost/_limits.erb'), - } - } # Template uses no variables concat::fragment { "${name}-file_footer": diff --git a/apache/manifests/vhost/custom.pp b/apache/manifests/vhost/custom.pp new file mode 100644 index 000000000..d85e4d091 --- /dev/null +++ b/apache/manifests/vhost/custom.pp @@ -0,0 +1,37 @@ +# See README.md for usage information +define apache::vhost::custom( + $content, + $ensure = 'present', + $priority = '25', +) { + include ::apache + + ## Apache include does not always work with spaces in the filename + $filename = regsubst($name, ' ', '_', 'G') + + ::apache::custom_config { $filename: + ensure => $ensure, + confdir => $::apache::vhost_dir, + content => $content, + priority => $priority, + } + + # NOTE(pabelanger): This code is duplicated in ::apache::vhost and needs to + # converted into something generic. + if $::apache::vhost_enable_dir { + $vhost_symlink_ensure = $ensure ? { + present => link, + default => $ensure, + } + + file { "${priority}-${filename}.conf symlink": + ensure => $vhost_symlink_ensure, + path => "${::apache::vhost_enable_dir}/${priority}-${filename}.conf", + target => "${::apache::vhost_dir}/${priority}-${filename}.conf", + owner => 'root', + group => $::apache::params::root_group, + mode => '0644', + require => Apache::Custom_config[$filename], + } + } +} diff --git a/apache/metadata.json b/apache/metadata.json index f06e6d529..85e688949 100644 --- a/apache/metadata.json +++ b/apache/metadata.json @@ -1,6 +1,6 @@ { "name": "puppetlabs-apache", - "version": "1.7.0", + "version": "1.7.1", "author": "puppetlabs", "summary": "Installs, configures, and manages Apache virtual hosts, web services, and modules.", "license": "Apache-2.0", @@ -68,7 +68,7 @@ "requirements": [ { "name": "pe", - "version_requirement": ">= 3.7.0 < 2015.3.0" + "version_requirement": ">= 3.7.0 < 2015.4.0" }, { "name": "puppet", diff --git a/apache/spec/acceptance/apache_parameters_spec.rb b/apache/spec/acceptance/apache_parameters_spec.rb index c46a22043..00e486fb6 100644 --- a/apache/spec/acceptance/apache_parameters_spec.rb +++ b/apache/spec/acceptance/apache_parameters_spec.rb @@ -1,7 +1,7 @@ require 'spec_helper_acceptance' require_relative './version.rb' -describe 'apache parameters', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache parameters' do # Currently this test only does something on FreeBSD. describe 'default_confd_files => false' do @@ -354,6 +354,20 @@ class { 'apache': end end + describe 'limitrequestfieldsize' do + describe 'setup' do + it 'applies cleanly' do + pp = "class { 'apache': limitreqfieldsize => '16830' }" + apply_manifest(pp, :catch_failures => true) + end + end + + describe file($conf_file) do + it { is_expected.to be_file } + it { is_expected.to contain 'LimitRequestFieldSize 16830' } + end + end + describe 'logging' do describe 'setup' do it 'applies cleanly' do diff --git a/apache/spec/acceptance/apache_ssl_spec.rb b/apache/spec/acceptance/apache_ssl_spec.rb index f8023fa24..ccf65c727 100644 --- a/apache/spec/acceptance/apache_ssl_spec.rb +++ b/apache/spec/acceptance/apache_ssl_spec.rb @@ -8,7 +8,7 @@ vhostd = '/etc/apache2/sites-available' end -describe 'apache ssl', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache ssl' do describe 'ssl parameters' do it 'runs without error' do diff --git a/apache/spec/acceptance/class_spec.rb b/apache/spec/acceptance/class_spec.rb index 0cf1c36f9..47b0d36fa 100644 --- a/apache/spec/acceptance/class_spec.rb +++ b/apache/spec/acceptance/class_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper_acceptance' -describe 'apache class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache class' do case fact('osfamily') when 'RedHat' package_name = 'httpd' diff --git a/apache/spec/acceptance/custom_config_spec.rb b/apache/spec/acceptance/custom_config_spec.rb index 683e6b648..fe73d111b 100644 --- a/apache/spec/acceptance/custom_config_spec.rb +++ b/apache/spec/acceptance/custom_config_spec.rb @@ -1,7 +1,7 @@ require 'spec_helper_acceptance' require_relative './version.rb' -describe 'apache::custom_config define', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::custom_config define' do context 'invalid config' do it 'should not add the config' do pp = <<-EOS diff --git a/apache/spec/acceptance/default_mods_spec.rb b/apache/spec/acceptance/default_mods_spec.rb index 0199f180b..c2d6a8c26 100644 --- a/apache/spec/acceptance/default_mods_spec.rb +++ b/apache/spec/acceptance/default_mods_spec.rb @@ -15,7 +15,7 @@ servicename = 'apache2' end -describe 'apache::default_mods class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::default_mods class' do describe 'no default mods' do # Using puppet_apply as a helper it 'should apply with no errors' do diff --git a/apache/spec/acceptance/itk_spec.rb b/apache/spec/acceptance/itk_spec.rb index 2dde8f407..c220a70f8 100644 --- a/apache/spec/acceptance/itk_spec.rb +++ b/apache/spec/acceptance/itk_spec.rb @@ -3,26 +3,41 @@ case fact('osfamily') when 'Debian' service_name = 'apache2' + majrelease = fact('operatingsystemmajrelease') + if [ '6', '7', '10.04', '12.04'].include?(majrelease) + variant = :itk_only + else + variant = :prefork + end when 'FreeBSD' service_name = 'apache24' -else - # Not implemented yet - service_name = :skip + majrelease = fact('operatingsystemmajrelease') + variant = :prefork end -describe 'apache::mod::itk class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) or service_name.equal? :skip do +describe 'apache::mod::itk class', :if => service_name do describe 'running puppet code' do # Using puppet_apply as a helper it 'should work with no errors' do - pp = <<-EOS - class { 'apache': - mpm_module => 'itk', - } - EOS + pp = case variant + when :prefork + <<-EOS + class { 'apache': + mpm_module => 'prefork', + } + class { 'apache::mod::itk': } + EOS + when :itk_only + <<-EOS + class { 'apache': + mpm_module => 'itk', + } + EOS + end # Run it twice and test for idempotency apply_manifest(pp, :catch_failures => true) - expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero + apply_manifest(pp, :catch_changes => true) end end diff --git a/apache/spec/acceptance/mod_fcgid_spec.rb b/apache/spec/acceptance/mod_fcgid_spec.rb index e99a7f299..ce3b5b5b2 100644 --- a/apache/spec/acceptance/mod_fcgid_spec.rb +++ b/apache/spec/acceptance/mod_fcgid_spec.rb @@ -1,7 +1,7 @@ require 'spec_helper_acceptance' -describe 'apache::mod::fcgid class', :unless => (UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) or (fact('operatingsystem') == 'OracleLinux' and fact('operatingsystemmajrelease') == '7')) do - context "default fcgid config", :if => (fact('osfamily') == 'RedHat' and fact('operatingsystemmajrelease') != '5') do +describe 'apache::mod::fcgid class', :if => ((fact('osfamily') == 'RedHat' and fact('operatingsystemmajrelease') != '5') and !(fact('operatingsystem') == 'OracleLinux' and fact('operatingsystemmajrelease') == '7')) do + context "default fcgid config" do it 'succeeds in puppeting fcgid' do pp = <<-EOS class { 'epel': } # mod_fcgid lives in epel diff --git a/apache/spec/acceptance/mod_mime_spec.rb b/apache/spec/acceptance/mod_mime_spec.rb index 89b3c800f..e47360b5e 100644 --- a/apache/spec/acceptance/mod_mime_spec.rb +++ b/apache/spec/acceptance/mod_mime_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper_acceptance' -describe 'apache::mod::mime class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::mod::mime class' do case fact('osfamily') when 'Debian' mod_dir = '/etc/apache2/mods-available' diff --git a/apache/spec/acceptance/mod_negotiation_spec.rb b/apache/spec/acceptance/mod_negotiation_spec.rb index a44edbf26..48eb896b6 100644 --- a/apache/spec/acceptance/mod_negotiation_spec.rb +++ b/apache/spec/acceptance/mod_negotiation_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper_acceptance' -describe 'apache::mod::negotiation class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::mod::negotiation class' do case fact('osfamily') when 'Debian' vhost_dir = '/etc/apache2/sites-enabled' diff --git a/apache/spec/acceptance/mod_pagespeed_spec.rb b/apache/spec/acceptance/mod_pagespeed_spec.rb index 5fa342e9a..f8060a167 100644 --- a/apache/spec/acceptance/mod_pagespeed_spec.rb +++ b/apache/spec/acceptance/mod_pagespeed_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper_acceptance' -describe 'apache::mod::pagespeed class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::mod::pagespeed class' do case fact('osfamily') when 'Debian' vhost_dir = '/etc/apache2/sites-enabled' diff --git a/apache/spec/acceptance/mod_passenger_spec.rb b/apache/spec/acceptance/mod_passenger_spec.rb index 5af9113e5..df9cd9e13 100644 --- a/apache/spec/acceptance/mod_passenger_spec.rb +++ b/apache/spec/acceptance/mod_passenger_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper_acceptance' -describe 'apache::mod::passenger class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::mod::passenger class' do case fact('osfamily') when 'Debian' service_name = 'apache2' diff --git a/apache/spec/acceptance/mod_php_spec.rb b/apache/spec/acceptance/mod_php_spec.rb index 502ec1ddc..a42f52373 100644 --- a/apache/spec/acceptance/mod_php_spec.rb +++ b/apache/spec/acceptance/mod_php_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper_acceptance' -describe 'apache::mod::php class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::mod::php class' do case fact('osfamily') when 'Debian' vhost_dir = '/etc/apache2/sites-enabled' diff --git a/apache/spec/acceptance/mod_proxy_html_spec.rb b/apache/spec/acceptance/mod_proxy_html_spec.rb index e45600089..840ea563f 100644 --- a/apache/spec/acceptance/mod_proxy_html_spec.rb +++ b/apache/spec/acceptance/mod_proxy_html_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper_acceptance' -describe 'apache::mod::proxy_html class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::mod::proxy_html class' do case fact('osfamily') when 'Debian' service_name = 'apache2' diff --git a/apache/spec/acceptance/mod_security_spec.rb b/apache/spec/acceptance/mod_security_spec.rb index 67ad7d5b8..4fcf0f551 100644 --- a/apache/spec/acceptance/mod_security_spec.rb +++ b/apache/spec/acceptance/mod_security_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper_acceptance' -describe 'apache::mod::security class', :unless => (UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) or (fact('osfamily') == 'Debian' and (fact('lsbdistcodename') == 'squeeze' or fact('lsbdistcodename') == 'lucid' or fact('lsbdistcodename') == 'precise' or fact('lsbdistcodename') == 'wheezy'))) do +describe 'apache::mod::security class', :unless => (fact('osfamily') == 'Debian' and (fact('lsbdistcodename') == 'squeeze' or fact('lsbdistcodename') == 'lucid' or fact('lsbdistcodename') == 'precise' or fact('lsbdistcodename') == 'wheezy')) do case fact('osfamily') when 'Debian' mod_dir = '/etc/apache2/mods-available' diff --git a/apache/spec/acceptance/mod_suphp_spec.rb b/apache/spec/acceptance/mod_suphp_spec.rb index 33f57fba6..da2c6042e 100644 --- a/apache/spec/acceptance/mod_suphp_spec.rb +++ b/apache/spec/acceptance/mod_suphp_spec.rb @@ -1,11 +1,9 @@ require 'spec_helper_acceptance' -describe 'apache::mod::suphp class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do - case fact('operatingsystem') - when 'Ubuntu' - context "default suphp config" do - it 'succeeds in puppeting suphp' do - pp = <<-EOS +describe 'apache::mod::suphp class', :if => fact('operatingsystem') == 'Ubuntu' do + context "default suphp config" do + it 'succeeds in puppeting suphp' do + pp = <<-EOS class { 'apache': mpm_module => 'prefork', } @@ -24,32 +22,32 @@ class { 'apache': } class { 'apache::mod::php': } class { 'apache::mod::suphp': } - EOS - apply_manifest(pp, :catch_failures => true) - end + EOS + apply_manifest(pp, :catch_failures => true) + end - describe service('apache2') do - it { is_expected.to be_enabled } - it { is_expected.to be_running } - end + describe service('apache2') do + it { is_expected.to be_enabled } + it { is_expected.to be_running } + end - it 'should answer to suphp.example.com' do - timeout = 0 - loop do - r = shell('curl suphp.example.com:80') - timeout += 1 - break if r.stdout =~ /^daemon$/ - if timeout > 40 - expect(timeout < 40).to be true - break - end - sleep(1) - end - shell("/usr/bin/curl suphp.example.com:80") do |r| - expect(r.stdout).to match(/^daemon$/) - expect(r.exit_code).to eq(0) - end + it 'should answer to suphp.example.com' do + timeout = 0 + loop do + r = shell('curl suphp.example.com:80') + timeout += 1 + break if r.stdout =~ /^daemon$/ + if timeout > 40 + expect(timeout < 40).to be true + break end + sleep(1) end + shell("/usr/bin/curl suphp.example.com:80") do |r| + expect(r.stdout).to match(/^daemon$/) + expect(r.exit_code).to eq(0) + end + end + end end diff --git a/apache/spec/acceptance/prefork_worker_spec.rb b/apache/spec/acceptance/prefork_worker_spec.rb index 0ac27232d..234b6acef 100644 --- a/apache/spec/acceptance/prefork_worker_spec.rb +++ b/apache/spec/acceptance/prefork_worker_spec.rb @@ -36,7 +36,7 @@ class { 'apache': end end -describe 'apache::mod::worker class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::mod::worker class' do describe 'running puppet code' do # Using puppet_apply as a helper it 'should work with no errors' do @@ -58,7 +58,7 @@ class { 'apache': end end -describe 'apache::mod::prefork class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::mod::prefork class' do describe 'running puppet code' do # Using puppet_apply as a helper it 'should work with no errors' do diff --git a/apache/spec/acceptance/service_spec.rb b/apache/spec/acceptance/service_spec.rb index b51ca386f..c3124c846 100644 --- a/apache/spec/acceptance/service_spec.rb +++ b/apache/spec/acceptance/service_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper_acceptance' -describe 'apache::service class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::service class' do describe 'adding dependencies in between the base class and service class' do it 'should work with no errors' do pp = <<-EOS diff --git a/apache/spec/acceptance/unsupported_spec.rb b/apache/spec/acceptance/unsupported_spec.rb deleted file mode 100644 index 085845dbf..000000000 --- a/apache/spec/acceptance/unsupported_spec.rb +++ /dev/null @@ -1,13 +0,0 @@ -require 'spec_helper_acceptance' - -describe 'unsupported distributions and OSes', :if => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do - it 'should fail' do - pp = <<-EOS - class { 'apache': } - apache::vhost { 'test.lan': - docroot => '/var/www', - } - EOS - expect(apply_manifest(pp, :expect_failures => true).stderr).to match(/unsupported/i) - end -end diff --git a/apache/spec/acceptance/vhost_spec.rb b/apache/spec/acceptance/vhost_spec.rb index 5fa91d5dc..5d0294130 100644 --- a/apache/spec/acceptance/vhost_spec.rb +++ b/apache/spec/acceptance/vhost_spec.rb @@ -1,7 +1,7 @@ require 'spec_helper_acceptance' require_relative './version.rb' -describe 'apache::vhost define', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do +describe 'apache::vhost define' do context 'no default vhosts' do it 'should create no default vhosts' do pp = <<-EOS @@ -202,8 +202,8 @@ class { 'apache': ip_based => true, docroot => '/var/www/html', } - host { 'ipv4.example.com': ip => '127.0.0.1', } - host { 'ipv6.example.com': ip => '127.0.0.2', } + host { 'host1.example.com': ip => '127.0.0.1', } + host { 'host2.example.com': ip => '127.0.0.2', } file { '/var/www/html/index.html': ensure => file, content => "Hello from vhost\\n", @@ -230,20 +230,20 @@ class { 'apache': it { is_expected.not_to contain 'NameVirtualHost 127.0.0.2:80' } end - it 'should answer to ipv4.example.com' do - shell("/usr/bin/curl ipv4.example.com:80", {:acceptable_exit_codes => 0}) do |r| + it 'should answer to host1.example.com' do + shell("/usr/bin/curl host1.example.com:80", {:acceptable_exit_codes => 0}) do |r| expect(r.stdout).to eq("Hello from vhost\n") end end - it 'should answer to ipv6.example.com' do - shell("/usr/bin/curl ipv6.example.com:80", {:acceptable_exit_codes => 0}) do |r| + it 'should answer to host2.example.com' do + shell("/usr/bin/curl host2.example.com:80", {:acceptable_exit_codes => 0}) do |r| expect(r.stdout).to eq("Hello from vhost\n") end end end - context 'new vhost with IPv6 address on port 80' do + context 'new vhost with IPv6 address on port 80', :ipv6 do it 'should configure one apache vhost with an ipv6 address' do pp = <<-EOS class { 'apache': @@ -505,8 +505,8 @@ class { 'apache': } case fact('lsbdistcodename') when 'precise', 'wheezy' - context 'vhost fallbackresource example' do - it 'should configure a vhost with Fallbackresource' do + context 'vhost FallbackResource example' do + it 'should configure a vhost with FallbackResource' do pp = <<-EOS class { 'apache': } apache::vhost { 'fallback.example.net': @@ -1232,54 +1232,58 @@ class { 'apache': } end describe 'wsgi' do - it 'import_script applies cleanly' do - pp = <<-EOS - class { 'apache': } - class { 'apache::mod::wsgi': } - host { 'test.server': ip => '127.0.0.1' } - apache::vhost { 'test.server': - docroot => '/tmp', - wsgi_application_group => '%{GLOBAL}', - wsgi_daemon_process => 'wsgi', - wsgi_daemon_process_options => {processes => '2'}, - wsgi_process_group => 'nobody', - wsgi_script_aliases => { '/test' => '/test1' }, - wsgi_pass_authorization => 'On', - } - EOS - apply_manifest(pp, :catch_failures => true) + context 'on lucid', :if => fact('lsbdistcodename') == 'lucid' do + it 'import_script applies cleanly' do + pp = <<-EOS + class { 'apache': } + class { 'apache::mod::wsgi': } + host { 'test.server': ip => '127.0.0.1' } + apache::vhost { 'test.server': + docroot => '/tmp', + wsgi_application_group => '%{GLOBAL}', + wsgi_daemon_process => 'wsgi', + wsgi_daemon_process_options => {processes => '2'}, + wsgi_process_group => 'nobody', + wsgi_script_aliases => { '/test' => '/test1' }, + wsgi_pass_authorization => 'On', + } + EOS + apply_manifest(pp, :catch_failures => true) + end end - it 'import_script applies cleanly', :unless => (fact('lsbdistcodename') == 'lucid' or UNSUPPORTED_PLATFORMS.include?(fact('osfamily'))) do - pp = <<-EOS - class { 'apache': } - class { 'apache::mod::wsgi': } - host { 'test.server': ip => '127.0.0.1' } - apache::vhost { 'test.server': - docroot => '/tmp', - wsgi_application_group => '%{GLOBAL}', - wsgi_daemon_process => 'wsgi', - wsgi_daemon_process_options => {processes => '2'}, - wsgi_import_script => '/test1', - wsgi_import_script_options => { application-group => '%{GLOBAL}', process-group => 'wsgi' }, - wsgi_process_group => 'nobody', - wsgi_script_aliases => { '/test' => '/test1' }, - wsgi_pass_authorization => 'On', - wsgi_chunked_request => 'On', - } - EOS - apply_manifest(pp, :catch_failures => true) - end + context 'on everything but lucid', :unless => fact('lsbdistcodename') == 'lucid' do + it 'import_script applies cleanly' do + pp = <<-EOS + class { 'apache': } + class { 'apache::mod::wsgi': } + host { 'test.server': ip => '127.0.0.1' } + apache::vhost { 'test.server': + docroot => '/tmp', + wsgi_application_group => '%{GLOBAL}', + wsgi_daemon_process => 'wsgi', + wsgi_daemon_process_options => {processes => '2'}, + wsgi_import_script => '/test1', + wsgi_import_script_options => { application-group => '%{GLOBAL}', process-group => 'wsgi' }, + wsgi_process_group => 'nobody', + wsgi_script_aliases => { '/test' => '/test1' }, + wsgi_pass_authorization => 'On', + wsgi_chunked_request => 'On', + } + EOS + apply_manifest(pp, :catch_failures => true) + end - describe file("#{$vhost_dir}/25-test.server.conf"), :unless => (fact('lsbdistcodename') == 'lucid' or UNSUPPORTED_PLATFORMS.include?(fact('osfamily'))) do - it { is_expected.to be_file } - it { is_expected.to contain 'WSGIApplicationGroup %{GLOBAL}' } - it { is_expected.to contain 'WSGIDaemonProcess wsgi processes=2' } - it { is_expected.to contain 'WSGIImportScript /test1 application-group=%{GLOBAL} process-group=wsgi' } - it { is_expected.to contain 'WSGIProcessGroup nobody' } - it { is_expected.to contain 'WSGIScriptAlias /test "/test1"' } - it { is_expected.to contain 'WSGIPassAuthorization On' } - it { is_expected.to contain 'WSGIChunkedRequest On' } + describe file("#{$vhost_dir}/25-test.server.conf") do + it { is_expected.to be_file } + it { is_expected.to contain 'WSGIApplicationGroup %{GLOBAL}' } + it { is_expected.to contain 'WSGIDaemonProcess wsgi processes=2' } + it { is_expected.to contain 'WSGIImportScript /test1 application-group=%{GLOBAL} process-group=wsgi' } + it { is_expected.to contain 'WSGIProcessGroup nobody' } + it { is_expected.to contain 'WSGIScriptAlias /test "/test1"' } + it { is_expected.to contain 'WSGIPassAuthorization On' } + it { is_expected.to contain 'WSGIChunkedRequest On' } + end end end @@ -1326,6 +1330,13 @@ class { 'apache': } describe 'fastcgi' do it 'applies cleanly' do pp = <<-EOS + if $::operatingsystem == 'Ubuntu' and versioncmp($::operatingsystemrelease, '10.04') >= 0 { + include ::apt + apt::ppa { 'multiverse': + before => Class['Apache::Mod::Fastcgi'], + } + } + class { 'apache': } class { 'apache::mod::fastcgi': } host { 'test.server': ip => '127.0.0.1' } diff --git a/apache/spec/classes/dev_spec.rb b/apache/spec/classes/dev_spec.rb index eb3d76593..933d67703 100644 --- a/apache/spec/classes/dev_spec.rb +++ b/apache/spec/classes/dev_spec.rb @@ -77,7 +77,6 @@ :osfamily => 'Gentoo', :operatingsystem => 'Gentoo', :operatingsystemrelease => '3.16.1-gentoo', - :concat_basedir => '/dne', :is_pe => false, :concat_basedir => '/foo', :id => 'root', diff --git a/apache/spec/classes/mod/itk_spec.rb b/apache/spec/classes/mod/itk_spec.rb index 164440876..cd2e6c67a 100644 --- a/apache/spec/classes/mod/itk_spec.rb +++ b/apache/spec/classes/mod/itk_spec.rb @@ -37,6 +37,10 @@ end context "with Apache version >= 2.4" do + let :pre_condition do + 'class { "apache": mpm_module => prefork, }' + end + let :params do { :apache_version => '2.4', @@ -52,6 +56,10 @@ end end context "on a FreeBSD OS" do + let :pre_condition do + 'class { "apache": mpm_module => false, }' + end + let :facts do { :osfamily => 'FreeBSD', diff --git a/apache/spec/classes/mod/ldap_spec.rb b/apache/spec/classes/mod/ldap_spec.rb new file mode 100644 index 000000000..2b82d8d1b --- /dev/null +++ b/apache/spec/classes/mod/ldap_spec.rb @@ -0,0 +1,78 @@ +require 'spec_helper' + +describe 'apache::mod::ldap', :type => :class do + let :pre_condition do + 'include apache' + end + + context "on a Debian OS" do + let :facts do + { + :lsbdistcodename => 'squeeze', + :osfamily => 'Debian', + :operatingsystemrelease => '6', + :concat_basedir => '/dne', + :id => 'root', + :kernel => 'Linux', + :operatingsystem => 'Debian', + :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', + :is_pe => false, + } + end + it { is_expected.to contain_class("apache::params") } + it { is_expected.to contain_class("apache::mod::ldap") } + it { is_expected.to contain_apache__mod('ldap') } + + context 'default ldap_trusted_global_cert_file' do + it { is_expected.to contain_file('ldap.conf').without_content(/^LDAPTrustedGlobalCert/) } + end + + context 'ldap_trusted_global_cert_file param' do + let(:params) { { :ldap_trusted_global_cert_file => 'ca.pem' } } + it { is_expected.to contain_file('ldap.conf').with_content(/^LDAPTrustedGlobalCert CA_BASE64 ca\.pem$/) } + end + + context 'ldap_trusted_global_cert_file and ldap_trusted_global_cert_type params' do + let(:params) {{ + :ldap_trusted_global_cert_file => 'ca.pem', + :ldap_trusted_global_cert_type => 'CA_DER' + }} + it { is_expected.to contain_file('ldap.conf').with_content(/^LDAPTrustedGlobalCert CA_DER ca\.pem$/) } + end + end #Debian + + context "on a RedHat OS" do + let :facts do + { + :osfamily => 'RedHat', + :operatingsystemrelease => '6', + :concat_basedir => '/dne', + :id => 'root', + :kernel => 'Linux', + :operatingsystem => 'RedHat', + :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', + :is_pe => false, + } + end + it { is_expected.to contain_class("apache::params") } + it { is_expected.to contain_class("apache::mod::ldap") } + it { is_expected.to contain_apache__mod('ldap') } + + context 'default ldap_trusted_global_cert_file' do + it { is_expected.to contain_file('ldap.conf').without_content(/^LDAPTrustedGlobalCert/) } + end + + context 'ldap_trusted_global_cert_file param' do + let(:params) { { :ldap_trusted_global_cert_file => 'ca.pem' } } + it { is_expected.to contain_file('ldap.conf').with_content(/^LDAPTrustedGlobalCert CA_BASE64 ca\.pem$/) } + end + + context 'ldap_trusted_global_cert_file and ldap_trusted_global_cert_type params' do + let(:params) {{ + :ldap_trusted_global_cert_file => 'ca.pem', + :ldap_trusted_global_cert_type => 'CA_DER' + }} + it { is_expected.to contain_file('ldap.conf').with_content(/^LDAPTrustedGlobalCert CA_DER ca\.pem$/) } + end + end # Redhat +end diff --git a/apache/spec/classes/mod/pagespeed_spec.rb b/apache/spec/classes/mod/pagespeed_spec.rb index c3f5a4144..44c60053e 100644 --- a/apache/spec/classes/mod/pagespeed_spec.rb +++ b/apache/spec/classes/mod/pagespeed_spec.rb @@ -21,7 +21,16 @@ it { is_expected.to contain_class("apache::params") } it { is_expected.to contain_apache__mod('pagespeed') } it { is_expected.to contain_package("mod-pagespeed-stable") } - it { is_expected.to contain_file('pagespeed.conf') } + + context "when setting additional_configuration to a Hash" do + let :params do { :additional_configuration => { 'Key' => 'Value' } } end + it { is_expected.to contain_file('pagespeed.conf').with_content /Key Value/ } + end + + context "when setting additional_configuration to an Array" do + let :params do { :additional_configuration => [ 'Key Value' ] } end + it { is_expected.to contain_file('pagespeed.conf').with_content /Key Value/ } + end end context "on a RedHat OS" do diff --git a/apache/spec/classes/mod/passenger_spec.rb b/apache/spec/classes/mod/passenger_spec.rb index c438241e0..34dcc5c6f 100644 --- a/apache/spec/classes/mod/passenger_spec.rb +++ b/apache/spec/classes/mod/passenger_spec.rb @@ -14,7 +14,6 @@ :lsbdistcodename => 'squeeze', :operatingsystem => 'Debian', :id => 'root', - :kernel => 'Linux', :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', :is_pe => false, } diff --git a/apache/spec/classes/mod/php_spec.rb b/apache/spec/classes/mod/php_spec.rb index 246b3bbd4..6814b8a8f 100644 --- a/apache/spec/classes/mod/php_spec.rb +++ b/apache/spec/classes/mod/php_spec.rb @@ -88,7 +88,7 @@ let :params do { :extensions => ['.php','.php5']} end - it { is_expected.to contain_file("php5.conf").with_content(/AddHandler php5-script .php .php5\n/) } + it { is_expected.to contain_file("php5.conf").with_content(Regexp.new(Regexp.escape(''))) } end context "with specific version" do let :pre_condition do diff --git a/apache/spec/defines/vhost_custom_spec.rb b/apache/spec/defines/vhost_custom_spec.rb new file mode 100644 index 000000000..804be86b8 --- /dev/null +++ b/apache/spec/defines/vhost_custom_spec.rb @@ -0,0 +1,99 @@ +require 'spec_helper' + +describe 'apache::vhost::custom', :type => :define do + let :title do + 'rspec.example.com' + end + let :default_params do + { + :content => 'foobar' + } + end + describe 'os-dependent items' do + context "on RedHat based systems" do + let :default_facts do + { + :osfamily => 'RedHat', + :operatingsystemrelease => '6', + :operatingsystem => 'RedHat', + :concat_basedir => '/dne', + :id => 'root', + :kernel => 'Linux', + :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', + :is_pe => false, + } + end + let :params do default_params end + let :facts do default_facts end + end + context "on Debian based systems" do + let :default_facts do + { + :osfamily => 'Debian', + :operatingsystemrelease => '6', + :lsbdistcodename => 'squeeze', + :operatingsystem => 'Debian', + :concat_basedir => '/dne', + :id => 'root', + :kernel => 'Linux', + :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', + :is_pe => false, + } + end + let :params do default_params end + let :facts do default_facts end + it { is_expected.to contain_file("apache_rspec.example.com").with( + :ensure => 'present', + :content => 'foobar', + :path => '/etc/apache2/sites-available/25-rspec.example.com.conf', + ) } + it { is_expected.to contain_file("25-rspec.example.com.conf symlink").with( + :ensure => 'link', + :path => '/etc/apache2/sites-enabled/25-rspec.example.com.conf', + :target => '/etc/apache2/sites-available/25-rspec.example.com.conf' + ) } + end + context "on FreeBSD systems" do + let :default_facts do + { + :osfamily => 'FreeBSD', + :operatingsystemrelease => '9', + :operatingsystem => 'FreeBSD', + :concat_basedir => '/dne', + :id => 'root', + :kernel => 'FreeBSD', + :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', + :is_pe => false, + } + end + let :params do default_params end + let :facts do default_facts end + it { is_expected.to contain_file("apache_rspec.example.com").with( + :ensure => 'present', + :content => 'foobar', + :path => '/usr/local/etc/apache24/Vhosts/25-rspec.example.com.conf', + ) } + end + context "on Gentoo systems" do + let :default_facts do + { + :osfamily => 'Gentoo', + :operatingsystem => 'Gentoo', + :operatingsystemrelease => '3.16.1-gentoo', + :concat_basedir => '/dne', + :id => 'root', + :kernel => 'Linux', + :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin', + :is_pe => false, + } + end + let :params do default_params end + let :facts do default_facts end + it { is_expected.to contain_file("apache_rspec.example.com").with( + :ensure => 'present', + :content => 'foobar', + :path => '/etc/apache2/vhosts.d/25-rspec.example.com.conf', + ) } + end + end +end diff --git a/apache/spec/defines/vhost_spec.rb b/apache/spec/defines/vhost_spec.rb index 3f5347254..9dd563a9c 100644 --- a/apache/spec/defines/vhost_spec.rb +++ b/apache/spec/defines/vhost_spec.rb @@ -154,7 +154,10 @@ 'ssl_verify_depth' => '3', 'ssl_options' => '+ExportCertData', 'ssl_openssl_conf_cmd' => 'DHParameters "foo.pem"', + 'ssl_proxy_check_peer_cn' => 'on', + 'ssl_proxy_check_peer_name' => 'on', 'ssl_proxyengine' => true, + 'priority' => '30', 'default_vhost' => true, 'servername' => 'example.com', @@ -323,8 +326,7 @@ 'krb_authoritative' => 'off', 'krb_auth_realms' => ['EXAMPLE.ORG','EXAMPLE.NET'], 'krb_5keytab' => '/tmp/keytab5', - 'krb_local_user_mapping' => 'off', - 'limit_request_field_size' => '54321', + 'krb_local_user_mapping' => 'off' } end let :facts do @@ -432,6 +434,13 @@ it { is_expected.to contain_concat__fragment('rspec.example.com-ssl') } it { is_expected.to contain_concat__fragment('rspec.example.com-ssl').with( :content => /^\s+SSLOpenSSLConfCmd\s+DHParameters "foo.pem"$/ ) } + it { is_expected.to contain_concat__fragment('rspec.example.com-sslproxy') } + it { is_expected.to contain_concat__fragment('rspec.example.com-sslproxy').with( + :content => /^\s+SSLProxyEngine On$/ ) } + it { is_expected.to contain_concat__fragment('rspec.example.com-sslproxy').with( + :content => /^\s+SSLProxyCheckPeerCN\s+on$/ ) } + it { is_expected.to contain_concat__fragment('rspec.example.com-sslproxy').with( + :content => /^\s+SSLProxyCheckPeerName\s+on$/ ) } it { is_expected.to contain_concat__fragment('rspec.example.com-suphp') } it { is_expected.to contain_concat__fragment('rspec.example.com-php_admin') } it { is_expected.to contain_concat__fragment('rspec.example.com-header') } @@ -462,8 +471,6 @@ :content => /^\s+KrbSaveCredentials\soff$/)} it { is_expected.to contain_concat__fragment('rspec.example.com-auth_kerb').with( :content => /^\s+KrbVerifyKDC\son$/)} - it { is_expected.to contain_concat__fragment('rspec.example.com-limits').with( - :content => /^\s+LimitRequestFieldSize\s54321$/)} end context 'vhost with multiple ip addresses' do let :params do @@ -674,6 +681,7 @@ it { is_expected.to_not contain_concat__fragment('rspec.example.com-serveralias') } it { is_expected.to_not contain_concat__fragment('rspec.example.com-setenv') } it { is_expected.to_not contain_concat__fragment('rspec.example.com-ssl') } + it { is_expected.to_not contain_concat__fragment('rspec.example.com-sslproxy') } it { is_expected.to_not contain_concat__fragment('rspec.example.com-suphp') } it { is_expected.to_not contain_concat__fragment('rspec.example.com-php_admin') } it { is_expected.to_not contain_concat__fragment('rspec.example.com-header') } @@ -696,6 +704,18 @@ it { is_expected.to compile } it { is_expected.not_to contain_concat__fragment('rspec.example.com-docroot') } end + context 'ssl_proxyengine without ssl' do + let :params do + { + 'docroot' => '/rspec/docroot', + 'ssl' => false, + 'ssl_proxyengine' => true, + } + end + it { is_expected.to compile } + it { is_expected.not_to contain_concat__fragment('rspec.example.com-ssl') } + it { is_expected.to contain_concat__fragment('rspec.example.com-sslproxy') } + end end describe 'access logs' do let :facts do diff --git a/apache/spec/spec_helper_acceptance.rb b/apache/spec/spec_helper_acceptance.rb index d767b1e60..6bfdefd71 100644 --- a/apache/spec/spec_helper_acceptance.rb +++ b/apache/spec/spec_helper_acceptance.rb @@ -4,9 +4,12 @@ run_puppet_install_helper -UNSUPPORTED_PLATFORMS = ['Suse','windows','AIX','Solaris'] - RSpec.configure do |c| + # apache on Ubuntu 10.04 and 12.04 doesn't like IPv6 VirtualHosts, so we skip ipv6 tests on those systems + if fact('operatingsystem') == 'Ubuntu' and (fact('operatingsystemrelease') == '10.04' or fact('operatingsystemrelease') == '12.04') + c.filter_run_excluding :ipv6 => true + end + # Project root proj_root = File.expand_path(File.join(File.dirname(__FILE__), '..')) @@ -24,27 +27,32 @@ apply_manifest_on(agents, pp, :catch_failures => false) end + if fact('osfamily') == 'Debian' + # Make sure snake-oil certs are installed. + shell 'apt-get install -y ssl-cert' + end + # Install module and dependencies hosts.each do |host| copy_module_to(host, :source => proj_root, :module_name => 'apache') + + on host, puppet('module','install','puppetlabs-stdlib') + on host, puppet('module','install','puppetlabs-concat', '--version 1.1.1', '--force') + # Required for mod_passenger tests. if fact('osfamily') == 'RedHat' - on host, puppet('module','install','stahnma/epel'), { :acceptable_exit_codes => [0,1] } - on host, puppet('module','install','puppetlabs/inifile'), { :acceptable_exit_codes => [0,1] } + on host, puppet('module','install','stahnma/epel') + on host, puppet('module','install','puppetlabs/inifile') end + # Required for manifest to make mod_pagespeed repository available if fact('osfamily') == 'Debian' - on host, puppet('module','install','puppetlabs-apt', '--version 1.8.0', '--force'), { :acceptable_exit_codes => [0,1] } - end - on host, puppet('module','install','puppetlabs-stdlib'), { :acceptable_exit_codes => [0,1] } - on host, puppet('module','install','puppetlabs-concat', '--version 1.1.1', '--force'), { :acceptable_exit_codes => [0,1] } - - # Make sure selinux is disabled before each test or apache won't work. - if ! UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) - on host, puppet('apply', '-e', - %{"exec { 'setenforce 0': path => '/bin:/sbin:/usr/bin:/usr/sbin', onlyif => 'which setenforce && getenforce | grep Enforcing', }"}), - { :acceptable_exit_codes => [0] } + on host, puppet('module','install','puppetlabs-apt', '--version 1.8.0', '--force') end + + # Make sure selinux is disabled so the tests work. + on host, puppet('apply', '-e', + %{"exec { 'setenforce 0': path => '/bin:/sbin:/usr/bin:/usr/sbin', onlyif => 'which setenforce && getenforce | grep Enforcing', }"}) end end end diff --git a/apache/templates/httpd.conf.erb b/apache/templates/httpd.conf.erb index 6fb989244..cc6998b9b 100644 --- a/apache/templates/httpd.conf.erb +++ b/apache/templates/httpd.conf.erb @@ -10,6 +10,7 @@ Timeout <%= @timeout %> KeepAlive <%= @keepalive %> MaxKeepAliveRequests <%= @max_keepalive_requests %> KeepAliveTimeout <%= @keepalive_timeout %> +LimitRequestFieldSize <%= @limitreqfieldsize %> <%- if @rewrite_lock and scope.function_versioncmp([@apache_version, '2.2']) <= 0 -%> RewriteLock <%= @rewrite_lock %> diff --git a/apache/templates/mod/fastcgi.conf.erb b/apache/templates/mod/fastcgi.conf.erb index 8d94a2361..93c8d86ab 100644 --- a/apache/templates/mod/fastcgi.conf.erb +++ b/apache/templates/mod/fastcgi.conf.erb @@ -1,6 +1,8 @@ # The Fastcgi Apache module configuration file is being # managed by Puppet and changes will be overwritten. - AddHandler fastcgi-script .fcgi + + SetHandler fastcgi-script + FastCgiIpcDir "<%= @fastcgi_lib_path %>" diff --git a/apache/templates/mod/ldap.conf.erb b/apache/templates/mod/ldap.conf.erb index 001977617..fbb4b9213 100644 --- a/apache/templates/mod/ldap.conf.erb +++ b/apache/templates/mod/ldap.conf.erb @@ -9,3 +9,6 @@ Satisfy all <%- end -%> +<% if @ldap_trusted_global_cert_file -%> +LDAPTrustedGlobalCert <%= @ldap_trusted_global_cert_type %> <%= @ldap_trusted_global_cert_file %> +<% end -%> diff --git a/apache/templates/mod/nss.conf.erb b/apache/templates/mod/nss.conf.erb index 22762ccac..b6ea50487 100644 --- a/apache/templates/mod/nss.conf.erb +++ b/apache/templates/mod/nss.conf.erb @@ -84,7 +84,7 @@ NSSRequireSafeNegotiation off ## SSL Virtual Host Context ## -> +> # General setup for the virtual host #DocumentRoot "/etc/httpd/htdocs" diff --git a/apache/templates/mod/pagespeed.conf.erb b/apache/templates/mod/pagespeed.conf.erb index a1b6f117a..051cf5bed 100644 --- a/apache/templates/mod/pagespeed.conf.erb +++ b/apache/templates/mod/pagespeed.conf.erb @@ -7,7 +7,7 @@ AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER application/xhtml+xml <% end -%> ModPagespeedFileCachePath "<%= @cache_path %>" ModPagespeedLogDir "<%= @log_dir %>" - + <% @memcache_servers.each do |server| -%> ModPagespeedMemcachedServers <%= server %> <% end -%> @@ -17,7 +17,7 @@ ModPagespeedRewriteLevel <%= @rewrite_level -%> <% @disable_filters.each do |filter| -%> ModPagespeedDisableFilters <%= filter %> <% end -%> - + <% @enable_filters.each do |filter| -%> ModPagespeedEnableFilters <%= filter %> <% end -%> @@ -93,6 +93,10 @@ ModPagespeedMessageBufferSize <%= @message_buffer_size %> SetHandler mod_pagespeed_message +<% if @additional_configuration.is_a? Array -%> +<%= @additional_configuration.join('\n') %> +<% else -%> <% @additional_configuration.each_pair do |key, value| -%> <%= key %> <%= value %> <% end -%> +<% end -%> diff --git a/apache/templates/mod/php5.conf.erb b/apache/templates/mod/php5.conf.erb index 44df2ae06..3fd100039 100644 --- a/apache/templates/mod/php5.conf.erb +++ b/apache/templates/mod/php5.conf.erb @@ -14,8 +14,9 @@ # # Cause the PHP interpreter to handle files with a .php extension. # -AddHandler php5-script <%= @extensions.flatten.compact.join(' ') %> -AddType text/html .php +)$"> + SetHandler php5-script + # # Add index.php to the list of files that will be served as directory diff --git a/apache/templates/vhost/_directories.erb b/apache/templates/vhost/_directories.erb index b1475513c..c95fda5a1 100644 --- a/apache/templates/vhost/_directories.erb +++ b/apache/templates/vhost/_directories.erb @@ -106,7 +106,9 @@ <%- end -%> <%- if directory['addhandlers'] and ! directory['addhandlers'].empty? -%> <%- [directory['addhandlers']].flatten.compact.each do |addhandler| -%> - AddHandler <%= addhandler['handler'] %> <%= Array(addhandler['extensions']).join(' ') %> + )$"> + SetHandler <%= addhandler['handler'] %> + <%- end -%> <%- end -%> <%- if directory['sethandler'] and directory['sethandler'] != '' -%> diff --git a/apache/templates/vhost/_limits.erb b/apache/templates/vhost/_limits.erb deleted file mode 100644 index 0bd56db35..000000000 --- a/apache/templates/vhost/_limits.erb +++ /dev/null @@ -1,5 +0,0 @@ - - ## Limit Request Values -<% if @limit_request_field_size -%> - LimitRequestFieldSize <%= @limit_request_field_size %> -<% end -%> diff --git a/apache/templates/vhost/_ssl.erb b/apache/templates/vhost/_ssl.erb index a3d76fb13..797435cc1 100644 --- a/apache/templates/vhost/_ssl.erb +++ b/apache/templates/vhost/_ssl.erb @@ -22,9 +22,6 @@ <%- if @ssl_crl_check && scope.function_versioncmp([@apache_version, '2.4']) >= 0 -%> SSLCARevocationCheck "<%= @ssl_crl_check %>" <%- end -%> - <%- if @ssl_proxyengine -%> - SSLProxyEngine On - <%- end -%> <%- if @ssl_protocol -%> SSLProtocol <%= [@ssl_protocol].flatten.compact.join(' ') %> <%- end -%> @@ -40,9 +37,6 @@ <%- if @ssl_verify_depth -%> SSLVerifyDepth <%= @ssl_verify_depth %> <%- end -%> - <%- if @ssl_proxy_machine_cert -%> - SSLProxyMachineCertificateFile "<%= @ssl_proxy_machine_cert %>" - <%- end -%> <%- if @ssl_options -%> SSLOptions <%= Array(@ssl_options).join(' ') %> <%- end -%> diff --git a/apache/templates/vhost/_sslproxy.erb b/apache/templates/vhost/_sslproxy.erb new file mode 100644 index 000000000..e58d52d0b --- /dev/null +++ b/apache/templates/vhost/_sslproxy.erb @@ -0,0 +1,14 @@ +<% if @ssl_proxyengine -%> + + # SSL Proxy directives + SSLProxyEngine On + <%- if @ssl_proxy_check_peer_cn -%> + SSLProxyCheckPeerCN <%= @ssl_proxy_check_peer_cn %> + <%- end -%> + <%- if @ssl_proxy_check_peer_name -%> + SSLProxyCheckPeerName <%= @ssl_proxy_check_peer_name %> + <%- end -%> + <%- if @ssl_proxy_machine_cert -%> + SSLProxyMachineCertificateFile "<%= @ssl_proxy_machine_cert %>" + <%- end -%> +<% end -%> diff --git a/ceilometer/examples/ceilometer_with_gnocchi.pp b/ceilometer/examples/ceilometer_with_gnocchi.pp new file mode 100644 index 000000000..c5a9533aa --- /dev/null +++ b/ceilometer/examples/ceilometer_with_gnocchi.pp @@ -0,0 +1,34 @@ +class { '::ceilometer': + metering_secret => 'secrete', + rabbit_userid => 'ceilometer', + rabbit_password => 'an_even_bigger_secret', + rabbit_host => '127.0.0.1', +} +class { '::ceilometer::db::mysql': + password => 'a_big_secret', +} +class { '::ceilometer::db': + database_connection => 'mysql://ceilometer:a_big_secret@127.0.0.1/ceilometer?charset=utf8', +} +class { '::ceilometer::keystone::auth': + password => 'a_big_secret', +} +class { '::ceilometer::client': } +class { '::ceilometer::expirer': } +class { '::ceilometer::agent::central': } +class { '::ceilometer::agent::notification': } +class { '::ceilometer::api': + enabled => true, + keystone_password => 'a_big_secret', + keystone_identity_uri => 'http://127.0.0.1:35357/', + service_name => 'httpd', +} +include ::apache +class { '::ceilometer::wsgi::apache': + ssl => false, +} + +class { '::ceilometer::collector': + meter_dispatcher => ['gnocchi'], +} +class { '::ceilometer::dispatcher::gnocchi': } diff --git a/ceilometer/manifests/collector.pp b/ceilometer/manifests/collector.pp index 8e22a93eb..bcddc7e46 100644 --- a/ceilometer/manifests/collector.pp +++ b/ceilometer/manifests/collector.pp @@ -22,12 +22,24 @@ # (optional) the ceilometer collector udp bind port. # Defaults to '4952' # +# [*meter_dispatcher*] +# (optional) dispatcher driver(s) to process meter data. +# Can be an array or a string. +# Defaults to 'database' +# +# [*event_dispatcher*] +# (optional) dispatcher driver(s) to process event data. +# Can be an array or a string. +# Defaults to 'database' +# class ceilometer::collector ( - $manage_service = true, - $enabled = true, - $package_ensure = 'present', - $udp_address = '0.0.0.0', - $udp_port = '4952', + $manage_service = true, + $enabled = true, + $package_ensure = 'present', + $udp_address = '0.0.0.0', + $udp_port = '4952', + $meter_dispatcher = 'database', + $event_dispatcher = 'database', ) { include ::ceilometer::params @@ -41,8 +53,10 @@ } ceilometer_config { - 'collector/udp_address' : value => $udp_address; - 'collector/udp_port' : value => $udp_port; + 'collector/udp_address': value => $udp_address; + 'collector/udp_port': value => $udp_port; + 'DEFAULT/meter_dispatcher': value => join(any2array($meter_dispatcher), ','); + 'DEFAULT/event_dispatcher': value => join(any2array($event_dispatcher), ','); } Package[$::ceilometer::params::collector_package_name] -> Service['ceilometer-collector'] diff --git a/ceilometer/manifests/dispatcher/gnocchi.pp b/ceilometer/manifests/dispatcher/gnocchi.pp new file mode 100644 index 000000000..4d348e890 --- /dev/null +++ b/ceilometer/manifests/dispatcher/gnocchi.pp @@ -0,0 +1,43 @@ +# Configure Gnocchi dispatcher for Ceilometer +# +# == Params +# +# [*filter_service_activity*] +# (optional) Filter out samples generated by Gnocchi service activity. +# Defaults to undef. +# +# [*filter_project*] +# (optional) Gnocchi project used to filter out samples +# generated by Gnocchi service activity +# Defaults to undef. +# +# [*url*] +# (optional) Gnocchi URL +# Defaults to undef. +# +# [*archive_policy*] +# (optional) The archive policy to use when the dispatcher +# Defaults to undef. +# +# [*resources_definition_file*] +# (optional) The Yaml file that defines mapping between samples +# and gnocchi resources/metrics. +# Defaults to undef. +# +class ceilometer::dispatcher::gnocchi ( + $filter_service_activity = undef, + $filter_project = undef, + $url = undef, + $archive_policy = undef, + $resources_definition_file = undef, +) { + + ceilometer_config { + 'dispatcher_gnocchi/filter_service_activity': value => $filter_service_activity; + 'dispatcher_gnocchi/filter_project': value => $filter_project; + 'dispatcher_gnocchi/url': value => $url; + 'dispatcher_gnocchi/archive_policy': value => $archive_policy; + 'dispatcher_gnocchi/resources_definition_file': value => $resources_definition_file; + } + +} diff --git a/ceilometer/spec/acceptance/ceilometer_with_mysql_spec.rb b/ceilometer/spec/acceptance/ceilometer_with_mysql_spec.rb index 5ef941cf3..27ce79745 100644 --- a/ceilometer/spec/acceptance/ceilometer_with_mysql_spec.rb +++ b/ceilometer/spec/acceptance/ceilometer_with_mysql_spec.rb @@ -83,6 +83,7 @@ class { '::ceilometer::api': keystone_password => 'a_big_secret', keystone_identity_uri => 'http://127.0.0.1:35357/', } + class { '::ceilometer::dispatcher::gnocchi': } EOS diff --git a/ceilometer/spec/acceptance/ceilometer_wsgi_apache_spec.rb b/ceilometer/spec/acceptance/ceilometer_wsgi_apache_spec.rb index 998fb914a..16e000ac0 100644 --- a/ceilometer/spec/acceptance/ceilometer_wsgi_apache_spec.rb +++ b/ceilometer/spec/acceptance/ceilometer_wsgi_apache_spec.rb @@ -60,6 +60,7 @@ class { '::ceilometer::api': class { '::ceilometer::wsgi::apache': ssl => false, } + class { '::ceilometer::dispatcher::gnocchi': } EOS diff --git a/ceilometer/spec/classes/ceilometer_collector_spec.rb b/ceilometer/spec/classes/ceilometer_collector_spec.rb index 00c83d4ff..b3bbfff80 100644 --- a/ceilometer/spec/classes/ceilometer_collector_spec.rb +++ b/ceilometer/spec/classes/ceilometer_collector_spec.rb @@ -53,6 +53,8 @@ it 'configures ceilometer-collector server' do is_expected.to contain_ceilometer_config('collector/udp_address').with_value( '0.0.0.0' ) is_expected.to contain_ceilometer_config('collector/udp_port').with_value( '4952' ) + is_expected.to contain_ceilometer_config('DEFAULT/meter_dispatcher').with_value( 'database' ) + is_expected.to contain_ceilometer_config('DEFAULT/event_dispatcher').with_value( 'database' ) end it 'installs ceilometer-collector package' do diff --git a/ceilometer/spec/classes/ceilometer_dispatcher_gnocchi_spec.rb b/ceilometer/spec/classes/ceilometer_dispatcher_gnocchi_spec.rb new file mode 100644 index 000000000..24c158ce7 --- /dev/null +++ b/ceilometer/spec/classes/ceilometer_dispatcher_gnocchi_spec.rb @@ -0,0 +1,54 @@ +require 'spec_helper' + +describe 'ceilometer::dispatcher::gnocchi' do + + let :pre_condition do + "class { 'ceilometer': metering_secret => 's3cr3t' }" + end + + let :params do + {} + end + + shared_examples_for 'ceilometer-gnocchi-dispatcher' do + it 'configures gnocchi dispatcher' do + is_expected.to contain_ceilometer_config('dispatcher_gnocchi/filter_service_activity').with_value(nil) + is_expected.to contain_ceilometer_config('dispatcher_gnocchi/filter_project').with_value(nil) + is_expected.to contain_ceilometer_config('dispatcher_gnocchi/url').with_value(nil) + is_expected.to contain_ceilometer_config('dispatcher_gnocchi/archive_policy').with_value(nil) + is_expected.to contain_ceilometer_config('dispatcher_gnocchi/resources_definition_file').with_value(nil) + end + + context 'when overriding parameters' do + before do + params.merge!(:filter_service_activity => false, + :filter_project => true, + :url => 'http://foo', + :archive_policy => 'high', + :resources_definition_file => 'foo') + end + it { is_expected.to contain_ceilometer_config('dispatcher_gnocchi/filter_service_activity').with_value('false') } + it { is_expected.to contain_ceilometer_config('dispatcher_gnocchi/filter_project').with_value('true') } + it { is_expected.to contain_ceilometer_config('dispatcher_gnocchi/url').with_value('http://foo') } + it { is_expected.to contain_ceilometer_config('dispatcher_gnocchi/archive_policy').with_value('high') } + it { is_expected.to contain_ceilometer_config('dispatcher_gnocchi/resources_definition_file').with_value('foo') } + end + end + + context 'on Debian platforms' do + let :facts do + { :osfamily => 'Debian' } + end + + it_configures 'ceilometer-gnocchi-dispatcher' + end + + context 'on RedHat platforms' do + let :facts do + { :osfamily => 'RedHat' } + end + + it_configures 'ceilometer-gnocchi-dispatcher' + end + +end diff --git a/ceph/.gitreview b/ceph/.gitreview index 206cfb9e0..9c7dba51a 100644 --- a/ceph/.gitreview +++ b/ceph/.gitreview @@ -1,4 +1,4 @@ [gerrit] host=review.openstack.org port=29418 -project=stackforge/puppet-ceph.git +project=openstack/puppet-ceph.git diff --git a/ceph/Puppetfile b/ceph/Puppetfile index f84861ad4..38e09fc04 100644 --- a/ceph/Puppetfile +++ b/ceph/Puppetfile @@ -2,7 +2,7 @@ forge "http://forge.puppetlabs.com" mod 'puppetlabs/stdlib', :git => 'git://github.com/puppetlabs/puppetlabs-stdlib', - :ref => 'origin/4.x' + :ref => 'origin/4.9.x' mod 'puppetlabs/apt', :git => 'git://github.com/puppetlabs/puppetlabs-apt', diff --git a/ceph/README.md b/ceph/README.md index 1d711e6df..a10d3ef71 100644 --- a/ceph/README.md +++ b/ceph/README.md @@ -45,7 +45,7 @@ Development ----------- ``` -git clone https://github.com/stackforge/puppet-ceph.git +git clone https://github.com/openstack/puppet-ceph.git cd puppet-ceph sudo gem install bundler bundle install @@ -188,13 +188,13 @@ chmod +x ./ci.sh GEM_HOME=~/.gems screen -dmS puppet-ceph gerritexec \ --timeout 14400 --hostname review.openstack.org \ --verbose --username puppetceph --script "../ci.sh > /tmp/out$$ 2>&1 ; r=$? ; pastebinit /tmp/out$$ ; exit $r #" \ - --project stackforge/puppet-ceph + --project openstack/puppet-ceph ``` Contributors ------------ -* https://github.com/stackforge/puppet-ceph/graphs/contributors +* https://github.com/openstack/puppet-ceph/graphs/contributors Release Notes ------------- diff --git a/ceph/Rakefile b/ceph/Rakefile index 8cac3021d..69a1e2ae5 100644 --- a/ceph/Rakefile +++ b/ceph/Rakefile @@ -10,4 +10,3 @@ end PuppetLint.configuration.fail_on_warnings = true PuppetLint.configuration.send('disable_80chars') -PuppetLint.configuration.send('disable_class_parameter_defaults') diff --git a/ceph/examples/common.yaml b/ceph/examples/common.yaml index 6ef85e388..4244dc063 100644 --- a/ceph/examples/common.yaml +++ b/ceph/examples/common.yaml @@ -44,6 +44,6 @@ ceph::profile::params::client_keys: ceph::profile::params::osds: '/dev/sdc': - journal: '/dev/sdb1' + journal: '/dev/sdb' '/dev/sdd': - journal: '/dev/sdb2' + journal: '/dev/sdb' diff --git a/ceph/examples/nodes/first.yaml b/ceph/examples/nodes/first.yaml index a57312071..02ea11dbb 100644 --- a/ceph/examples/nodes/first.yaml +++ b/ceph/examples/nodes/first.yaml @@ -2,5 +2,5 @@ ######## OSD ceph::profile::params::osds: '/dev/sdb': - journal: '/tmp/journal' + journal: '/srv/journal' diff --git a/ceph/examples/nodes/second.yaml b/ceph/examples/nodes/second.yaml index a57312071..02ea11dbb 100644 --- a/ceph/examples/nodes/second.yaml +++ b/ceph/examples/nodes/second.yaml @@ -2,5 +2,5 @@ ######## OSD ceph::profile::params::osds: '/dev/sdb': - journal: '/tmp/journal' + journal: '/srv/journal' diff --git a/ceph/manifests/repo.pp b/ceph/manifests/repo.pp index f3a4e1a83..5f791efa4 100644 --- a/ceph/manifests/repo.pp +++ b/ceph/manifests/repo.pp @@ -180,7 +180,7 @@ name => 'ext-ceph-fastcgi', baseurl => "http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel${el}-x86_64-basic/ref/master", gpgcheck => '1', - gpgkey => 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/autobuild.asc', + gpgkey => 'https://git.ceph.com/autobuild.asc', mirrorlist => absent, priority => '20', # prefer ceph repos over EPEL tag => 'ceph', diff --git a/ceph/metadata.json b/ceph/metadata.json index 8fee7ef4b..fd79dc1c4 100644 --- a/ceph/metadata.json +++ b/ceph/metadata.json @@ -1,10 +1,10 @@ { - "name": "stackforge-ceph", + "name": "openstack-ceph", "version": "1.0.0", - "author": "Puppet Ceph community and StackForge Contributors", + "author": "Puppet Ceph community and OpenStack (StackForge) Contributors", "summary": "Community Developed Ceph Module", "license": "Apache-2.0", - "source": "git://github.com/stackforge/puppet-ceph.git", + "source": "git://github.com/openstack/puppet-ceph.git", "project_page": "https://launchpad.net/puppet-ceph", "issues_url": "https://bugs.launchpad.net/puppet-ceph", "requirements": [ diff --git a/ceph/spec/acceptance/ceph_rgw_spec.rb b/ceph/spec/acceptance/ceph_rgw_spec.rb index 1f242ab7c..590026792 100644 --- a/ceph/spec/acceptance/ceph_rgw_spec.rb +++ b/ceph/spec/acceptance/ceph_rgw_spec.rb @@ -102,6 +102,7 @@ class { 'ceph::repo': class { 'ceph': fsid => '#{fsid}', mon_host => $::ipaddress, + mon_initial_members => 'a', osd_pool_default_size => '1', osd_pool_default_min_size => '1', } @@ -201,10 +202,6 @@ class { 'ceph': Keystone_user_role<||> -> Ceph::Rgw::Keystone['radosgw.gateway'] EOS - apply_manifest(pp, :catch_failures => true) - # Enable as soon as remaining changes are fixed - #apply_manifest(pp, :catch_changes => true) - osfamily = fact 'osfamily' servicequery = { @@ -212,14 +209,21 @@ class { 'ceph': 'RedHat' => 'service ceph-radosgw status id=radosgw.gateway', } - shell servicequery[osfamily] do |r| - expect(r.exit_code).to be_zero - end + # RGW on CentOS is currently broken, so lets disable tests for now. + if osfamily != 'RedHat' + apply_manifest(pp, :catch_failures => true) + # Enable as soon as remaining changes are fixed + #apply_manifest(pp, :catch_changes => true) + + shell servicequery[osfamily] do |r| + expect(r.exit_code).to be_zero + end - shell "swift -V 2.0 -A http://127.0.0.1:5000/v2.0 -U #{test_tenant}:#{test_user} -K #{test_password} stat" do |r| - expect(r.exit_code).to be_zero - expect(r.stdout).to match(/Content-Type: text\/plain; charset=utf-8/) - expect(r.stdout).not_to match(/401 Unauthorized/) + shell "swift -V 2.0 -A http://127.0.0.1:5000/v2.0 -U #{test_tenant}:#{test_user} -K #{test_password} stat" do |r| + expect(r.exit_code).to be_zero + expect(r.stdout).to match(/Content-Type: text\/plain; charset=utf-8/) + expect(r.stdout).not_to match(/401 Unauthorized/) + end end end @@ -275,7 +279,12 @@ class { 'apache': } EOS - apply_manifest(purge, :catch_failures => true) + osfamily = fact 'osfamily' + + # RGW on CentOS is currently broken, so lets disable tests for now. + if osfamily != 'RedHat' + apply_manifest(purge, :catch_failures => true) + end end end end diff --git a/ceph/spec/classes/ceph_osds_spec.rb b/ceph/spec/classes/ceph_osds_spec.rb index 7721e8d40..33ba80282 100644 --- a/ceph/spec/classes/ceph_osds_spec.rb +++ b/ceph/spec/classes/ceph_osds_spec.rb @@ -24,7 +24,7 @@ { :args => { '/dev/sdb' => { - 'journal' => '/tmp/journal', + 'journal' => '/srv/journal', }, '/srv/data' => { }, @@ -38,7 +38,7 @@ it { is_expected.to contain_ceph__osd('/dev/sdb').with( :ensure => 'present', - :journal => '/tmp/journal', + :journal => '/srv/journal', :cluster => 'CLUSTER' ) is_expected.to contain_ceph__osd('/srv/data').with( diff --git a/ceph/spec/classes/ceph_profile_osd_spec.rb b/ceph/spec/classes/ceph_profile_osd_spec.rb index 0b309bdd3..aaa175655 100644 --- a/ceph/spec/classes/ceph_profile_osd_spec.rb +++ b/ceph/spec/classes/ceph_profile_osd_spec.rb @@ -27,8 +27,8 @@ end it { is_expected.to contain_class('ceph::profile::client') } - it { is_expected.to contain_ceph__osd('/dev/sdc').with(:journal => '/dev/sdb1') } - it { is_expected.to contain_ceph__osd('/dev/sdd').with(:journal => '/dev/sdb2') } + it { is_expected.to contain_ceph__osd('/dev/sdc').with(:journal => '/dev/sdb') } + it { is_expected.to contain_ceph__osd('/dev/sdd').with(:journal => '/dev/sdb') } end context 'with the host specific first.yaml' do @@ -38,7 +38,7 @@ end it { is_expected.to contain_class('ceph::profile::client') } - it { is_expected.to contain_ceph__osd('/dev/sdb').with( :journal => '/tmp/journal') } + it { is_expected.to contain_ceph__osd('/dev/sdb').with( :journal => '/srv/journal') } end end diff --git a/ceph/spec/classes/ceph_repo_spec.rb b/ceph/spec/classes/ceph_repo_spec.rb index b6678f64e..4ff9dd820 100644 --- a/ceph/spec/classes/ceph_repo_spec.rb +++ b/ceph/spec/classes/ceph_repo_spec.rb @@ -332,7 +332,7 @@ :name => 'ext-ceph-fastcgi', :baseurl => 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel6-x86_64-basic/ref/master', :gpgcheck => '1', - :gpgkey => 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/autobuild.asc', + :gpgkey => 'https://git.ceph.com/autobuild.asc', :mirrorlist => 'absent', :priority => '20' ) } @@ -440,7 +440,7 @@ :name => 'ext-ceph-fastcgi', :baseurl => 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel6-x86_64-basic/ref/master', :gpgcheck => '1', - :gpgkey => 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/autobuild.asc', + :gpgkey => 'https://git.ceph.com/autobuild.asc', :mirrorlist => 'absent', :priority => '20' ) } @@ -588,7 +588,7 @@ :name => 'ext-ceph-fastcgi', :baseurl => 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel7-x86_64-basic/ref/master', :gpgcheck => '1', - :gpgkey => 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/autobuild.asc', + :gpgkey => 'https://git.ceph.com/autobuild.asc', :mirrorlist => 'absent', :priority => '20' ) } @@ -686,7 +686,7 @@ :name => 'ext-ceph-fastcgi', :baseurl => 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel7-x86_64-basic/ref/master', :gpgcheck => '1', - :gpgkey => 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/autobuild.asc', + :gpgkey => 'https://git.ceph.com/autobuild.asc', :mirrorlist => 'absent', :priority => '20' ) } diff --git a/ceph/spec/defines/ceph_osd_spec.rb b/ceph/spec/defines/ceph_osd_spec.rb index cf57ab8f1..01ff1e509 100644 --- a/ceph/spec/defines/ceph_osd_spec.rb +++ b/ceph/spec/defines/ceph_osd_spec.rb @@ -26,39 +26,39 @@ describe "with default params" do let :title do - '/tmp' + '/srv' end - it { is_expected.to contain_exec('ceph-osd-prepare-/tmp').with( + it { is_expected.to contain_exec('ceph-osd-prepare-/srv').with( 'command' => "/bin/true # comment to satisfy puppet syntax requirements set -ex -if ! test -b /tmp ; then - mkdir -p /tmp +if ! test -b /srv ; then + mkdir -p /srv fi -ceph-disk prepare /tmp +ceph-disk prepare /srv ", 'unless' => "/bin/true # comment to satisfy puppet syntax requirements set -ex -ceph-disk list | grep -E ' */tmp1? .*ceph data, (prepared|active)' || -ls -l /var/lib/ceph/osd/ceph-* | grep ' /tmp\$' +ceph-disk list | grep -E ' */srv1? .*ceph data, (prepared|active)' || +ls -l /var/lib/ceph/osd/ceph-* | grep ' /srv\$' ", 'logoutput' => true ) } - it { is_expected.to contain_exec('ceph-osd-activate-/tmp').with( + it { is_expected.to contain_exec('ceph-osd-activate-/srv').with( 'command' => "/bin/true # comment to satisfy puppet syntax requirements set -ex -if ! test -b /tmp ; then - mkdir -p /tmp +if ! test -b /srv ; then + mkdir -p /srv fi # activate happens via udev when using the entire device -if ! test -b /tmp || ! test -b /tmp1 ; then - ceph-disk activate /tmp || true +if ! test -b /srv || ! test -b /srv1 ; then + ceph-disk activate /srv || true fi ", 'unless' => "/bin/true # comment to satisfy puppet syntax requirements set -ex -ceph-disk list | grep -E ' */tmp1? .*ceph data, active' || -ls -ld /var/lib/ceph/osd/ceph-* | grep ' /tmp\$' +ceph-disk list | grep -E ' */srv1? .*ceph data, active' || +ls -ld /var/lib/ceph/osd/ceph-* | grep ' /srv\$' ", 'logoutput' => true ) } @@ -67,46 +67,46 @@ describe "with custom params" do let :title do - '/tmp/data' + '/srv/data' end let :params do { :cluster => 'testcluster', - :journal => '/tmp/journal', + :journal => '/srv/journal', } end - it { is_expected.to contain_exec('ceph-osd-prepare-/tmp/data').with( + it { is_expected.to contain_exec('ceph-osd-prepare-/srv/data').with( 'command' => "/bin/true # comment to satisfy puppet syntax requirements set -ex -if ! test -b /tmp/data ; then - mkdir -p /tmp/data +if ! test -b /srv/data ; then + mkdir -p /srv/data fi -ceph-disk prepare --cluster testcluster /tmp/data /tmp/journal +ceph-disk prepare --cluster testcluster /srv/data /srv/journal ", 'unless' => "/bin/true # comment to satisfy puppet syntax requirements set -ex -ceph-disk list | grep -E ' */tmp/data1? .*ceph data, (prepared|active)' || -ls -l /var/lib/ceph/osd/testcluster-* | grep ' /tmp/data\$' +ceph-disk list | grep -E ' */srv/data1? .*ceph data, (prepared|active)' || +ls -l /var/lib/ceph/osd/testcluster-* | grep ' /srv/data\$' ", 'logoutput' => true ) } - it { is_expected.to contain_exec('ceph-osd-activate-/tmp/data').with( + it { is_expected.to contain_exec('ceph-osd-activate-/srv/data').with( 'command' => "/bin/true # comment to satisfy puppet syntax requirements set -ex -if ! test -b /tmp/data ; then - mkdir -p /tmp/data +if ! test -b /srv/data ; then + mkdir -p /srv/data fi # activate happens via udev when using the entire device -if ! test -b /tmp/data || ! test -b /tmp/data1 ; then - ceph-disk activate /tmp/data || true +if ! test -b /srv/data || ! test -b /srv/data1 ; then + ceph-disk activate /srv/data || true fi ", 'unless' => "/bin/true # comment to satisfy puppet syntax requirements set -ex -ceph-disk list | grep -E ' */tmp/data1? .*ceph data, active' || -ls -ld /var/lib/ceph/osd/testcluster-* | grep ' /tmp/data\$' +ceph-disk list | grep -E ' */srv/data1? .*ceph data, active' || +ls -ld /var/lib/ceph/osd/testcluster-* | grep ' /srv/data\$' ", 'logoutput' => true ) } @@ -115,7 +115,7 @@ describe "with ensure absent" do let :title do - '/tmp' + '/srv' end let :params do @@ -124,14 +124,14 @@ } end - it { is_expected.to contain_exec('remove-osd-/tmp').with( + it { is_expected.to contain_exec('remove-osd-/srv').with( 'command' => "/bin/true # comment to satisfy puppet syntax requirements set -ex if [ -z \"\$id\" ] ; then - id=\$(ceph-disk list | sed -nEe 's:^ */tmp1? .*(ceph data|mounted on).*osd\\.([0-9]+).*:\\2:p') + id=\$(ceph-disk list | sed -nEe 's:^ */srv1? .*(ceph data|mounted on).*osd\\.([0-9]+).*:\\2:p') fi if [ -z \"\$id\" ] ; then - id=\$(ls -ld /var/lib/ceph/osd/ceph-* | sed -nEe 's:.*/ceph-([0-9]+) *-> */tmp\$:\\1:p' || true) + id=\$(ls -ld /var/lib/ceph/osd/ceph-* | sed -nEe 's:.*/ceph-([0-9]+) *-> */srv\$:\\1:p' || true) fi if [ \"\$id\" ] ; then stop ceph-osd cluster=ceph id=\$id || true @@ -146,10 +146,10 @@ 'unless' => "/bin/true # comment to satisfy puppet syntax requirements set -ex if [ -z \"\$id\" ] ; then - id=\$(ceph-disk list | sed -nEe 's:^ */tmp1? .*(ceph data|mounted on).*osd\\.([0-9]+).*:\\2:p') + id=\$(ceph-disk list | sed -nEe 's:^ */srv1? .*(ceph data|mounted on).*osd\\.([0-9]+).*:\\2:p') fi if [ -z \"\$id\" ] ; then - id=\$(ls -ld /var/lib/ceph/osd/ceph-* | sed -nEe 's:.*/ceph-([0-9]+) *-> */tmp\$:\\1:p' || true) + id=\$(ls -ld /var/lib/ceph/osd/ceph-* | sed -nEe 's:.*/ceph-([0-9]+) *-> */srv\$:\\1:p' || true) fi if [ \"\$id\" ] ; then test ! -d /var/lib/ceph/osd/ceph-\$id diff --git a/ceph/spec/defines/ceph_rgw_apache_spec.rb b/ceph/spec/defines/ceph_rgw_apache_spec.rb index 28f93c626..8e4c0c754 100644 --- a/ceph/spec/defines/ceph_rgw_apache_spec.rb +++ b/ceph/spec/defines/ceph_rgw_apache_spec.rb @@ -126,13 +126,14 @@ class { 'ceph::repo': let :facts do { - :concat_basedir => '/var/lib/puppet/concat', - :fqdn => 'myhost.domain', - :hostname => 'myhost', - :osfamily => 'RedHat', - :operatingsystem => 'RedHat', - :operatingsystemrelease => '6', - :pkg_fastcgi => 'libapache2-mod-fastcgi', + :concat_basedir => '/var/lib/puppet/concat', + :fqdn => 'myhost.domain', + :hostname => 'myhost', + :osfamily => 'RedHat', + :operatingsystem => 'RedHat', + :operatingsystemrelease => '6', + :operatingsystemmajrelease => '6', + :pkg_fastcgi => 'libapache2-mod-fastcgi', } end diff --git a/ceph/spec/defines/ceph_rgw_keystone_spec.rb b/ceph/spec/defines/ceph_rgw_keystone_spec.rb index 303a4096e..0e8f1d474 100644 --- a/ceph/spec/defines/ceph_rgw_keystone_spec.rb +++ b/ceph/spec/defines/ceph_rgw_keystone_spec.rb @@ -141,13 +141,14 @@ class { 'ceph::repo': extras => true, fastcgi => true, } let :facts do { - :concat_basedir => '/var/lib/puppet/concat', - :fqdn => 'myhost.domain', - :hostname => 'myhost', - :lsbdistcodename => 'Final', - :osfamily => 'RedHat', - :operatingsystem => 'RedHat', - :operatingsystemrelease => '6', + :concat_basedir => '/var/lib/puppet/concat', + :fqdn => 'myhost.domain', + :hostname => 'myhost', + :lsbdistcodename => 'Final', + :osfamily => 'RedHat', + :operatingsystem => 'RedHat', + :operatingsystemrelease => '6', + :operatingsystemmajrelease => '6', } end diff --git a/ceph/spec/fixtures/hieradata/common.yaml b/ceph/spec/fixtures/hieradata/common.yaml index b20c9e6f9..784591418 100644 --- a/ceph/spec/fixtures/hieradata/common.yaml +++ b/ceph/spec/fixtures/hieradata/common.yaml @@ -41,6 +41,6 @@ ceph::profile::params::client_keys: cap_osd: 'allow class-read object_prefix rbd_children, allow rwx pool=volumes' ceph::profile::params::osds: '/dev/sdc': - journal: '/dev/sdb1' + journal: '/dev/sdb' '/dev/sdd': - journal: '/dev/sdb2' + journal: '/dev/sdb' diff --git a/ceph/spec/fixtures/hieradata/nodes/first.yaml b/ceph/spec/fixtures/hieradata/nodes/first.yaml index a57312071..02ea11dbb 100644 --- a/ceph/spec/fixtures/hieradata/nodes/first.yaml +++ b/ceph/spec/fixtures/hieradata/nodes/first.yaml @@ -2,5 +2,5 @@ ######## OSD ceph::profile::params::osds: '/dev/sdb': - journal: '/tmp/journal' + journal: '/srv/journal' diff --git a/ceph/spec/spec_helper_acceptance.rb b/ceph/spec/spec_helper_acceptance.rb index 8c7158fc7..1639f3706 100644 --- a/ceph/spec/spec_helper_acceptance.rb +++ b/ceph/spec/spec_helper_acceptance.rb @@ -45,7 +45,7 @@ on host, puppet('module install puppetlabs/apt --version ">=1.4.0 <2.0.0"'), { :acceptable_exit_codes => [0,1] } on host, puppet('module install puppetlabs/concat --version ">=1.2.1 <2.0.0"'), { :acceptable_exit_codes => [0,1] } on host, puppet('module install puppetlabs/apache --version ">=1.4.1 <2.0.0"'), { :acceptable_exit_codes => [0,1] } - on host, puppet('module install stackforge/keystone --version ">=5.1.0 <6.0.0"'), { :acceptable_exit_codes => [0,1] } + on host, puppet('module install stackforge/keystone --version ">=5.1.0 <6.0.0"'), { :acceptable_exit_codes => [0,1] } # keystone >=5.1.0 <6.0.0 is not present in openstack/keystone puppet_module_install(:source => proj_root, :module_name => 'ceph') # Flush the firewall flushfw = <<-EOS diff --git a/ceph/spec/spec_helper_system.rb b/ceph/spec/spec_helper_system.rb index 695d12cd8..a64826b88 100644 --- a/ceph/spec/spec_helper_system.rb +++ b/ceph/spec/spec_helper_system.rb @@ -55,7 +55,7 @@ shell(:command => 'puppet module install --version ">=1.4.1 <2.0.0" puppetlabs/apache', :node => vm) shell(:command => 'puppet module install --version ">=5.1.0 <6.0.0" stackforge/keystone', - :node => vm) + :node => vm) # keystone >=5.1.0 <6.0.0 is not present in openstack/keystone rcp(:sp => File.join(proj_root, 'spec/fixtures/hieradata/hiera.yaml'), :dp => '/etc/puppet/hiera.yaml', :d => node(:name => vm)) diff --git a/ceph/spec/system/ceph_osd_spec.rb b/ceph/spec/system/ceph_osd_spec.rb index c839c4598..b613abdf6 100644 --- a/ceph/spec/system/ceph_osd_spec.rb +++ b/ceph/spec/system/ceph_osd_spec.rb @@ -228,7 +228,7 @@ class { 'ceph': authentication_type => 'none', } ceph::osd { '#{data}': - journal => '/tmp/journal' + journal => '/srv/journal' } EOS diff --git a/ceph/spec/system/ceph_profile_osd_spec.rb b/ceph/spec/system/ceph_profile_osd_spec.rb index b9fea63a2..75906b97f 100644 --- a/ceph/spec/system/ceph_profile_osd_spec.rb +++ b/ceph/spec/system/ceph_profile_osd_spec.rb @@ -135,7 +135,7 @@ class { 'ceph::repo': ceph::profile::params::authentication_type: 'none' ceph::profile::params::osds: '/dev/sdb': - journal: '/tmp/journal' + journal: '/srv/journal' EOS file = Tempfile.new('hieradata') diff --git a/ceph/spec/unit/provider/ceph_config/ini_setting_spec.rb b/ceph/spec/unit/provider/ceph_config/ini_setting_spec.rb index 8a944c691..e2b84cb23 100644 --- a/ceph/spec/unit/provider/ceph_config/ini_setting_spec.rb +++ b/ceph/spec/unit/provider/ceph_config/ini_setting_spec.rb @@ -15,7 +15,7 @@ # Author: Andrew Woodward # This is aparently one of the few ways to do this load -# see https://github.com/stackforge/puppet-nova/blob/master/spec/unit/provider/nova_config/ini_setting_spec.rb +# see https://github.com/openstack/puppet-nova/blob/master/spec/unit/provider/nova_config/ini_setting_spec.rb $LOAD_PATH.push( File.join( File.dirname(__FILE__), diff --git a/contrail/README.md b/contrail/README.md index e56867435..b3da49dba 100644 --- a/contrail/README.md +++ b/contrail/README.md @@ -1,6 +1,6 @@ # Puppet-contrail -[![Build Status](https://travis-ci.org/enovance/puppet-contrail.png?branch=master)](https://travis-ci.org/enovance/puppet-contrail) +[![Build Status](https://travis-ci.org/redhat-cip/puppet-contrail.png?branch=master)](https://travis-ci.org/redhat-cip/puppet-contrail) [![Puppet Forge](http://img.shields.io/puppetforge/v/eNovance/contrail.svg)](https://forge.puppetlabs.com/eNovance/contrail) [![License](http://img.shields.io/:license-apache2-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0.html) diff --git a/contrail/manifests/control/config.pp b/contrail/manifests/control/config.pp index c49b934d4..22013d28b 100644 --- a/contrail/manifests/control/config.pp +++ b/contrail/manifests/control/config.pp @@ -41,6 +41,16 @@ create_resources('contrail_control_config', $control_config) create_resources('contrail_control_nodemgr_config', $control_nodemgr_config) + if $forwarder { + if is_array($forwarder) { + $forwarders_option = join([join($forwarder, ';'),';'], '') + } else { + $forwarders_option = "${forwarder};" + } + } else { + $forwarders_option = '' + } + file { '/etc/contrail/dns/contrail-named.conf' : ensure => file, content => template('contrail/contrail-named.conf.erb'), diff --git a/contrail/metadata.json b/contrail/metadata.json index 5084ad44f..b43de8378 100644 --- a/contrail/metadata.json +++ b/contrail/metadata.json @@ -4,9 +4,9 @@ "author": "Yanis Guenane, Sebastien Badia, and eNovance Contributors", "summary": "Puppet module for Juniper OpenContrail", "license": "Apache-2.0", - "source": "git://github.com/enovance/puppet-{{cookiecutter.project_name}}.git", - "project_page": "https://github.com/enovance/puppet-contrail", - "issues_url": "https://github.com/enovance/puppet-contrail/issues", + "source": "git://github.com/redhat-cip/puppet-contrail.git", + "project_page": "https://github.com/redhat-cip/puppet-contrail", + "issues_url": "https://github.com/redhat-cip/puppet-contrail/issues", "description": "Installs and configures OpenContrail.", "operatingsystem_support": [ { diff --git a/contrail/templates/contrail-named.conf.erb b/contrail/templates/contrail-named.conf.erb index 4f065c553..bf6d70c08 100644 --- a/contrail/templates/contrail-named.conf.erb +++ b/contrail/templates/contrail-named.conf.erb @@ -35,5 +35,5 @@ view "_default_view_" { match-clients {any;}; match-destinations {any;}; match-recursive-only no; - forwarders {<%= @forwarder -%>; }; + forwarders { <%= @forwarders_option -%> }; }; diff --git a/elasticsearch/.fixtures.yml b/elasticsearch/.fixtures.yml index abc7962dc..d750f45cd 100644 --- a/elasticsearch/.fixtures.yml +++ b/elasticsearch/.fixtures.yml @@ -5,7 +5,7 @@ fixtures: branch: '4.7.x' apt: repo: 'https://github.com/puppetlabs/puppetlabs-apt.git' - branch: '1.8.x' + branch: '2.2.x' zypprepo: https://github.com/deadpoint/puppet-zypprepo.git yum: https://github.com/CERIT-SC/puppet-yum.git datacat: https://github.com/richardc/puppet-datacat.git diff --git a/elasticsearch/.pmtignore b/elasticsearch/.pmtignore index cc245cafa..b5e79edf0 100644 --- a/elasticsearch/.pmtignore +++ b/elasticsearch/.pmtignore @@ -1,2 +1,6 @@ spec/ Rakefile +junit/ +logs/ +Gemfile +Gemfile.lock diff --git a/elasticsearch/CHANGELOG.md b/elasticsearch/CHANGELOG.md index 3914fa878..3e1bcdb30 100644 --- a/elasticsearch/CHANGELOG.md +++ b/elasticsearch/CHANGELOG.md @@ -1,3 +1,30 @@ +##0.10.0 ( Dec 14, 2015 ) + +###Summary +Module now works with ES 2.x completely + +####Features +* Work with ES 2.x new plugin system and remain to work with 1.x +* Implemented datacat module from Richard Clamp so other modules can hook into it for adding configuration options +* Fixed init and systemd files to work with 1.x and 2.x +* Made the module work with newer pl-apt module versions +* Export es_include so it is passed on to ES +* Ability to supply long gpg key for apt repo + +####Bugfixes +* Documentation and typographical fixes +* Do not force puppet:/// schema resource +* Use package resource defaults rather than setting provider and source + +####Changes + +####Testing changes +* Improve unit testing and shorten the runtime + +####Known bugs +* Possible package conflicts when using ruby/python defines with main package name + + ##0.9.9 ( Sep 01, 2015 ) ###Summary diff --git a/elasticsearch/Gemfile b/elasticsearch/Gemfile index fdcc1a1d9..68c056e84 100644 --- a/elasticsearch/Gemfile +++ b/elasticsearch/Gemfile @@ -3,7 +3,7 @@ source 'https://rubygems.org' puppetversion = ENV['PUPPET_VERSION'] || '~> 3.8.0' gem 'puppet', puppetversion, :require => false -gem 'beaker', '2.27.0' +gem 'beaker' gem 'beaker-rspec' gem 'metadata-json-lint' gem 'rspec-puppet', '2.2.0' diff --git a/elasticsearch/README.md b/elasticsearch/README.md index 4f861f935..80737b620 100644 --- a/elasticsearch/README.md +++ b/elasticsearch/README.md @@ -1,7 +1,5 @@ #Elasticsearch Puppet module -[![Build Status](https://travis-ci.org/elastic/puppet-elasticsearch.png?branch=master)](https://travis-ci.org/elastic/puppet-elasticsearch) - ####Table of Contents 1. [Overview](#overview) @@ -25,7 +23,7 @@ This module manages Elasticsearch (http://www.elasticsearch.org/overview/elastic The elasticsearch module sets up Elasticsearch instances and can manage plugins and templates. -This module has been tested against all versions of ES 1.x (ES 1.0 - 1.7). **The module is not yet ready for ES 2.x, but it should be soon!** +This module has been tested against all versions of ES 1.x and 2.x ##Setup @@ -48,7 +46,7 @@ This module has been tested against all versions of ES 1.x (ES 1.0 - 1.7). **The #### Repository management When using the repository management you will need the following dependency modules: -* Debian/Ubuntu: [Puppetlabs/apt](http://forge.puppetlabs.com/puppetlabs/apt) Version 1.8.x or lower. +* Debian/Ubuntu: [Puppetlabs/apt](http://forge.puppetlabs.com/puppetlabs/apt) * OpenSuSE: [Darin/zypprepo](https://forge.puppetlabs.com/darin/zypprepo) ##Usage @@ -162,6 +160,11 @@ elasticsearch::plugin { 'elasticsearch/elasticsearch-cloud-aws/2.4.1': Please note that this does not work when you specify 'latest' as a version number. +####ES 2.x official plugins +For the Elasticsearch commercial plugins you can refer them to the simple name. + +See the [Plugin installation](https://www.elastic.co/guide/en/elasticsearch/plugins/current/installation.html) for more details. + ###Scripts Install [scripts](http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting.html) to be used by Elasticsearch. diff --git a/elasticsearch/lib/puppet/provider/elasticsearch_plugin/plugin.rb b/elasticsearch/lib/puppet/provider/elasticsearch_plugin/plugin.rb new file mode 100644 index 000000000..4c8bb2913 --- /dev/null +++ b/elasticsearch/lib/puppet/provider/elasticsearch_plugin/plugin.rb @@ -0,0 +1,130 @@ +$LOAD_PATH.unshift(File.join(File.dirname(__FILE__),"..","..","..")) + +Puppet::Type.type(:elasticsearch_plugin).provide(:plugin) do + desc "A provider for the resource type `elasticsearch_plugin`, + which handles plugin installation" + + commands :plugin => '/usr/share/elasticsearch/bin/plugin' + commands :es => '/usr/share/elasticsearch/bin/elasticsearch' + + def exists? + es_version + if !File.exists?(pluginfile) + debug "Plugin file #{pluginfile} does not exist" + return false + elsif File.exists?(pluginfile) && readpluginfile != pluginfile_content + debug "Got #{readpluginfile} Expected #{pluginfile_content}. Removing for reinstall" + self.destroy + return false + else + debug "Plugin exists" + return true + end + end + + def pluginfile_content + return @resource[:name] if is1x? + + if @resource[:name].split("/").count == 1 # Official plugin + version = plugin_version(@resource[:name]) + return "#{@resource[:name]}/#{version}" + else + return @resource[:name] + end + end + + def pluginfile + File.join(@resource[:plugin_dir], plugin_name(@resource[:name]), '.name') + end + + def writepluginfile + File.open(pluginfile, 'w') do |file| + file.write pluginfile_content + end + end + + def readpluginfile + f = File.open(pluginfile) + f.readline + end + + def install1x + if !@resource[:url].nil? + commands = [ plugin_name(@resource[:name]), '--url', @resource[:url] ] + elsif !@resource[:source].nil? + commands = [ plugin_name(@resource[:name]), '--url', "file://#{@resource[:source]}" ] + else + commands = [ @resource[:name] ] + end + commands + end + + def install2x + if !@resource[:url].nil? + commands = [ @resource[:url] ] + elsif !@resource[:source].nil? + commands = [ "file://#{@resource[:source]}" ] + else + commands = [ @resource[:name] ] + end + commands + end + + def create + es_version + commands = [] + commands << @resource[:proxy_args] if @resource[:proxy_args] + commands << 'install' + commands << install1x if is1x? + commands << install2x if is2x? + + plugin(commands) + writepluginfile + end + + def destroy + plugin(['remove', @resource[:name]]) + end + + def es_version + return @es_version if @es_version + begin + version = es('-v') # ES 1.x + rescue + version = es('--version') # ES 2.x + rescue + raise "Unknown ES version. Got #{version.inspect}" + ensure + @es_version = version.scan(/\d+\.\d+\.\d+(?:\-\S+)?/).first + debug "Found ES version #{@es_version}" + end + end + + def is1x? + Puppet::Util::Package.versioncmp(@es_version, '2.0.0') < 0 + end + + def is2x? + (Puppet::Util::Package.versioncmp(@es_version, '2.0.0') >= 0) && (Puppet::Util::Package.versioncmp(@es_version, '3.0.0') < 0) + end + + def plugin_version(plugin_name) + vendor, plugin, version = plugin_name.split('/') + return @es_version if is2x? && version.nil? + return version.scan(/\d+\.\d+\.\d+(?:\-\S+)?/).first unless version.nil? + return false + end + + def plugin_name(plugin_name) + + vendor, plugin, version = plugin_name.split('/') + + endname = vendor if plugin.nil? # If its a single name plugin like the ES 2.x official plugins + endname = plugin.gsub(/(elasticsearch-|es-)/, '') unless plugin.nil? + + return endname.downcase if is2x? + return endname + + end + +end diff --git a/elasticsearch/lib/puppet/type/elasticsearch_plugin.rb b/elasticsearch/lib/puppet/type/elasticsearch_plugin.rb new file mode 100644 index 000000000..145880f28 --- /dev/null +++ b/elasticsearch/lib/puppet/type/elasticsearch_plugin.rb @@ -0,0 +1,31 @@ +Puppet::Type.newtype(:elasticsearch_plugin) do + + @doc = "Plugin installation type" + + ensurable do + defaultvalues + defaultto :present + end + + newparam(:name, :namevar => true) do + desc 'An arbitrary name used as the identity of the resource.' + end + + newparam(:url) do + desc 'Url of the package' + end + + newparam(:source) do + desc 'Source of the package. puppet:// or file:// resource' + end + + newparam(:proxy_args) do + desc 'Proxy Host' + end + + newparam(:plugin_dir) do + desc 'Plugin directory' + defaultto '/usr/share/elasticsearch/plugins' + end + +end diff --git a/elasticsearch/manifests/config.pp b/elasticsearch/manifests/config.pp index 405c65341..a7b5d830b 100644 --- a/elasticsearch/manifests/config.pp +++ b/elasticsearch/manifests/config.pp @@ -65,11 +65,6 @@ mode => '0755', } - file { $elasticsearch::plugindir: - ensure => 'directory', - recurse => true, - } - file { $elasticsearch::datadir: ensure => 'directory', } @@ -100,6 +95,7 @@ } } + file { "${elasticsearch::params::homedir}/templates_import": ensure => 'directory', mode => '0644', @@ -133,7 +129,13 @@ } } elsif ( $elasticsearch::ensure == 'absent' ) { - # don't remove anything for now + + file { $elasticsearch::plugindir: + ensure => 'absent', + force => true, + backup => false, + } + } } diff --git a/elasticsearch/manifests/init.pp b/elasticsearch/manifests/init.pp index 724386144..768cf4c6f 100644 --- a/elasticsearch/manifests/init.pp +++ b/elasticsearch/manifests/init.pp @@ -93,7 +93,7 @@ # # [*proxy_url*] # For http and https downloads you can set a proxy server to use -# Format: proto://[user:pass@]server[:port]/ +# Format: proto://[user:pass@]server[:port]/ # Defaults to: undef (proxy disabled) # # [*elasticsearch_user*] @@ -136,6 +136,14 @@ # [*repo_version*] # Our repositories are versioned per major version (0.90, 1.0) select here which version you want # +# [*repo_key_id*] +# String. The apt GPG key id +# Default: D88E42B4 +# +# [*repo_key_source*] +# String. URL of the apt GPG key +# Default: http://packages.elastic.co/GPG-KEY-elasticsearch +# # [*logging_config*] # Hash representation of information you want in the logging.yml file # @@ -228,6 +236,8 @@ $java_package = undef, $manage_repo = false, $repo_version = undef, + $repo_key_id = 'D88E42B4', + $repo_key_source = 'http://packages.elastic.co/GPG-KEY-elasticsearch', $logging_file = undef, $logging_config = undef, $logging_template = undef, @@ -302,13 +312,13 @@ case $::osfamily { 'RedHat', 'Linux', 'Suse': { if ($version =~ /.+-\d/) { - $real_version = $version + $pkg_version = $version } else { - $real_version = "${version}-1" + $pkg_version = "${version}-1" } } default: { - $real_version = $version + $pkg_version = $version } } } @@ -398,6 +408,7 @@ Anchor['elasticsearch::begin'] -> Class['elasticsearch::package'] -> Class['elasticsearch::config'] + -> Elasticsearch::Plugin <| |> -> Elasticsearch::Instance <| |> -> Elasticsearch::Template <| |> diff --git a/elasticsearch/manifests/instance.pp b/elasticsearch/manifests/instance.pp index 75380a1d5..f1e14fcdc 100644 --- a/elasticsearch/manifests/instance.pp +++ b/elasticsearch/manifests/instance.pp @@ -211,6 +211,7 @@ owner => $elasticsearch::elasticsearch_user, group => undef, mode => '0644', + recurse => true, require => [ Exec["mkdir_datadir_elasticsearch_${name}"], Class['elasticsearch::package'] ], before => Elasticsearch::Service[$name], } diff --git a/elasticsearch/manifests/package.pp b/elasticsearch/manifests/package.pp index f53d9ef3a..5e3caa9f5 100644 --- a/elasticsearch/manifests/package.pp +++ b/elasticsearch/manifests/package.pp @@ -33,9 +33,27 @@ #### Package management + # set params: in operation if $elasticsearch::ensure == 'present' { + # Create directory to place the package file + exec { 'create_package_dir_elasticsearch': + cwd => '/', + path => ['/usr/bin', '/bin'], + command => "mkdir -p ${elasticsearch::package_dir}", + creates => $elasticsearch::package_dir, + } + + file { $elasticsearch::package_dir: + ensure => 'directory', + purge => $elasticsearch::purge_package_dir, + force => $elasticsearch::purge_package_dir, + backup => false, + require => Exec['create_package_dir_elasticsearch'], + } + + # Check if we want to install a specific version or not if $elasticsearch::version == false { @@ -47,7 +65,7 @@ } else { # install specific version - $package_ensure = $elasticsearch::real_version + $package_ensure = $elasticsearch::pkg_version } @@ -61,22 +79,6 @@ $package_dir = $elasticsearch::package_dir - # Create directory to place the package file - exec { 'create_package_dir_elasticsearch': - cwd => '/', - path => ['/usr/bin', '/bin'], - command => "mkdir -p ${elasticsearch::package_dir}", - creates => $elasticsearch::package_dir, - } - - file { $package_dir: - ensure => 'directory', - purge => $elasticsearch::purge_package_dir, - force => $elasticsearch::purge_package_dir, - backup => false, - require => Exec['create_package_dir_elasticsearch'], - } - $filenameArray = split($elasticsearch::package_url, '/') $basefilename = $filenameArray[-1] @@ -160,15 +162,6 @@ } $package_ensure = 'purged' - $package_dir = $elasticsearch::package_dir - - file { $package_dir: - ensure => 'absent', - purge => true, - force => true, - backup => false, - } - } if ($elasticsearch::package_provider == 'package') { diff --git a/elasticsearch/manifests/plugin.pp b/elasticsearch/manifests/plugin.pp index 47ce52419..b515a4906 100644 --- a/elasticsearch/manifests/plugin.pp +++ b/elasticsearch/manifests/plugin.pp @@ -79,25 +79,13 @@ include elasticsearch - Exec { - path => [ '/bin', '/usr/bin', '/usr/sbin', '/usr/local/bin', '/usr/local/sbin' ], - cwd => '/', - user => $elasticsearch::elasticsearch_user, - tries => 6, - try_sleep => 10, - timeout => 600, - } - $notify_service = $elasticsearch::restart_on_change ? { false => undef, default => Elasticsearch::Service[$instances], } if ($module_dir != undef) { - warning("module_dir settings is deprecated for plugin ${name}. The directory is now auto detected.") - $plugin_dir = $module_dir - } else { - $plugin_dir = plugin_dir($name) + warning("module_dir settings is deprecated for plugin ${name}. The directory is now auto detected.") } # set proxy by override or parse and use proxy_url from @@ -116,7 +104,7 @@ } } else { - $proxy = '' # lint:ignore:empty_string_assignment + $proxy = undef } if ($source != undef) { @@ -124,53 +112,32 @@ $filenameArray = split($source, '/') $basefilename = $filenameArray[-1] - file { "/tmp/${basefilename}": + $file_source = "${elasticsearch::package_dir}/${basefilename}" + + file { $file_source: ensure => 'file', source => $source, } - $real_url = "file:///tmp/${basefilename}" } elsif ($url != undef) { validate_string($url) - $real_url = $url - } else { - $real_url = undef - } - - if ($real_url == undef) { - $install_cmd = "${elasticsearch::plugintool}${proxy} install ${name}" - $exec_rets = [0,] - } else { - $install_cmd = "${elasticsearch::plugintool}${proxy} install ${name} --url ${real_url}" - $exec_rets = [0,1] } case $ensure { 'installed', 'present': { - $name_file_path = "${elasticsearch::plugindir}/${plugin_dir}/.name" - exec {"purge_plugin_${plugin_dir}_old": - command => "${elasticsearch::plugintool} --remove ${plugin_dir}", - onlyif => "test -e ${elasticsearch::plugindir}/${plugin_dir} && test \"$(cat ${name_file_path})\" != '${name}'", - before => Exec["install_plugin_${name}"], - } - exec {"install_plugin_${name}": - command => $install_cmd, - creates => "${elasticsearch::plugindir}/${plugin_dir}", - returns => $exec_rets, - notify => $notify_service, - require => File[$elasticsearch::plugindir], - } - file {$name_file_path: - ensure => file, - content => "${name}", # lint:ignore:only_variable_string - require => Exec["install_plugin_${name}"], + + elasticsearch_plugin { $name: + ensure => 'present', + source => $file_source, + url => $url, + proxy_args => $proxy, + notify => $notify_service, } + } 'absent': { - exec {"remove_plugin_${name}": - command => "${elasticsearch::plugintool} --remove ${plugin_dir}", - onlyif => "test -d ${elasticsearch::plugindir}/${plugin_dir}", - notify => $notify_service, + elasticsearch_plugin { $name: + ensure => absent, } } default: { diff --git a/elasticsearch/manifests/repo.pp b/elasticsearch/manifests/repo.pp index e2ab6ccb3..36cd549ef 100644 --- a/elasticsearch/manifests/repo.pp +++ b/elasticsearch/manifests/repo.pp @@ -39,8 +39,8 @@ location => "http://packages.elastic.co/elasticsearch/${elasticsearch::repo_version}/debian", release => 'stable', repos => 'main', - key => 'D88E42B4', - key_source => 'http://packages.elastic.co/GPG-KEY-elasticsearch', + key => $::elasticsearch::repo_key_id, + key_source => $::elasticsearch::repo_key_source, include_src => false, } } @@ -49,14 +49,14 @@ descr => 'elasticsearch repo', baseurl => "http://packages.elastic.co/elasticsearch/${elasticsearch::repo_version}/centos", gpgcheck => 1, - gpgkey => 'http://packages.elastic.co/GPG-KEY-elasticsearch', + gpgkey => $::elasticsearch::repo_key_source, enabled => 1, } } 'Suse': { exec { 'elasticsearch_suse_import_gpg': - command => 'rpmkeys --import http://packages.elastic.co/GPG-KEY-elasticsearch', - unless => 'test $(rpm -qa gpg-pubkey | grep -i "D88E42B4" | wc -l) -eq 1 ', + command => "rpmkeys --import ${::elasticsearch::repo_key_source}", + unless => "test $(rpm -qa gpg-pubkey | grep -i '${::elasticsearch::repo_key_id}' | wc -l) -eq 1 ", notify => [ Zypprepo['elasticsearch'] ], } @@ -66,7 +66,7 @@ autorefresh => 1, name => 'elasticsearch', gpgcheck => 1, - gpgkey => 'http://packages.elastic.co/GPG-KEY-elasticsearch', + gpgkey => $::elasticsearch::repo_key_source, type => 'yum', } } @@ -76,30 +76,31 @@ } # Package pinning - if ($elasticsearch::package_pin == true and $elasticsearch::version != false) { + case $::osfamily { 'Debian': { - if !defined(Class['apt']) { - class { 'apt': } - } + include ::apt - apt::pin { $elasticsearch::package_name: - ensure => 'present', - packages => $elasticsearch::package_name, - version => $elasticsearch::real_version, - priority => 1000, + if ($elasticsearch::package_pin == true and $elasticsearch::version != false) { + apt::pin { $elasticsearch::package_name: + ensure => 'present', + packages => $elasticsearch::package_name, + version => $elasticsearch::version, + priority => 1000, + } } + } 'RedHat', 'Linux': { - yum::versionlock { "0:elasticsearch-${elasticsearch::real_version}.noarch": - ensure => 'present', + if ($elasticsearch::package_pin == true and $elasticsearch::version != false) { + yum::versionlock { "0:elasticsearch-${elasticsearch::pkg_version}.noarch": + ensure => 'present', + } } } default: { warning("Unable to pin package for OSfamily \"${::osfamily}\".") } } - } - } diff --git a/elasticsearch/metadata.json b/elasticsearch/metadata.json index a13025ff4..0b76368e4 100644 --- a/elasticsearch/metadata.json +++ b/elasticsearch/metadata.json @@ -1,6 +1,6 @@ { "name": "elasticsearch-elasticsearch", - "version": "0.9.9", + "version": "0.10.0", "source": "https://github.com/elastic/puppet-elasticsearch", "author": "elasticsearch", "license": "Apache-2.0", diff --git a/elasticsearch/spec/acceptance/004_plugin_spec.rb b/elasticsearch/spec/acceptance/004_plugin_spec.rb index 19d79ab3d..be5f33387 100644 --- a/elasticsearch/spec/acceptance/004_plugin_spec.rb +++ b/elasticsearch/spec/acceptance/004_plugin_spec.rb @@ -251,4 +251,49 @@ end + describe "install via url" do + it 'Should run succesful' do + pp = "class { 'elasticsearch': config => { 'node.name' => 'elasticsearch001', 'cluster.name' => '#{test_settings['cluster_name']}' }, manage_repo => true, repo_version => '#{test_settings['repo_version']}', java_install => true } + elasticsearch::instance { 'es-01': config => { 'node.name' => 'elasticsearch001', 'http.port' => '#{test_settings['port_a']}' } } + elasticsearch::plugin{'HQ': url => 'https://github.com/royrusso/elasticsearch-HQ/archive/v2.0.3.zip', instances => 'es-01' } + " + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero + + end + + it 'make sure elasticsearch reports it as existing' do + curl_with_retries('validated plugin as installed', default, "http://localhost:#{test_settings['port_a']}/_nodes/?plugin | grep HQ", 0) + end + + end + + describe "module removal" do + + it 'should run successfully' do + pp = "class { 'elasticsearch': ensure => 'absent' } + elasticsearch::instance{ 'es-01': ensure => 'absent' } + " + + apply_manifest(pp, :catch_failures => true) + end + + describe file('/etc/elasticsearch/es-01') do + it { should_not be_directory } + end + + describe package(test_settings['package_name']) do + it { should_not be_installed } + end + + describe service(test_settings['service_name_a']) do + it { should_not be_enabled } + it { should_not be_running } + end + + end + + end diff --git a/elasticsearch/spec/acceptance/005_datapath_spec.rb b/elasticsearch/spec/acceptance/009_datapath_spec.rb similarity index 100% rename from elasticsearch/spec/acceptance/005_datapath_spec.rb rename to elasticsearch/spec/acceptance/009_datapath_spec.rb diff --git a/elasticsearch/spec/acceptance/021_es2x_spec.rb b/elasticsearch/spec/acceptance/021_es2x_spec.rb new file mode 100644 index 000000000..f119facaf --- /dev/null +++ b/elasticsearch/spec/acceptance/021_es2x_spec.rb @@ -0,0 +1,351 @@ +require 'spec_helper_acceptance' + +describe "elasticsearch 2x:" do + + shell("mkdir -p #{default['distmoduledir']}/another/files") + shell("cp /tmp/elasticsearch-kopf.zip #{default['distmoduledir']}/another/files/elasticsearch-kopf.zip") + + describe "Install a plugin from official repository" do + + it 'should run successfully' do + pp = "class { 'elasticsearch': config => { 'node.name' => 'elasticsearch001', 'cluster.name' => '#{test_settings['cluster_name']}' }, manage_repo => true, repo_version => '#{test_settings['repo_version2x']}', java_install => true, version => '2.0.0' } + elasticsearch::instance { 'es-01': config => { 'node.name' => 'elasticsearch001', 'http.port' => '#{test_settings['port_a']}' } } + elasticsearch::plugin{'lmenezes/elasticsearch-kopf': instances => 'es-01' } + " + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero + end + + describe service(test_settings['service_name_a']) do + it { should be_enabled } + it { should be_running } + end + + describe package(test_settings['package_name']) do + it { should be_installed } + end + + describe file(test_settings['pid_file_a']) do + it { should be_file } + its(:content) { should match /[0-9]+/ } + end + + it 'make sure the directory exists' do + shell('ls /usr/share/elasticsearch/plugins/kopf/', {:acceptable_exit_codes => 0}) + end + + it 'make sure elasticsearch reports it as existing' do + curl_with_retries('validated plugin as installed', default, "http://localhost:#{test_settings['port_a']}/_nodes/?plugin | grep kopf", 0) + end + + end + describe "Install a plugin from custom git repo" do + it 'should run successfully' do + end + + it 'make sure the directory exists' do + end + + it 'make sure elasticsearch reports it as existing' do + end + + end + + if fact('puppetversion') =~ /3\.[2-9]\./ + + describe "Install a non existing plugin" do + + it 'should run successfully' do + pp = "class { 'elasticsearch': config => { 'node.name' => 'elasticearch001', 'cluster.name' => '#{test_settings['cluster_name']}' }, manage_repo => true, repo_version => '#{test_settings['repo_version2x']}', java_install => true, version => '2.0.0' } + elasticsearch::instance { 'es-01': config => { 'node.name' => 'elasticsearch001', 'http.port' => '#{test_settings['port_a']}' } } + elasticsearch::plugin{'elasticsearch/non-existing': module_dir => 'non-existing', instances => 'es-01' } + " + # Run it twice and test for idempotency + apply_manifest(pp, :expect_failures => true) + end + + end + + else + # The exit codes have changes since Puppet 3.2x + # Since beaker expectations are based on the most recent puppet code All runs on previous versions fails. + end + + describe "module removal" do + + it 'should run successfully' do + pp = "class { 'elasticsearch': ensure => 'absent' } + elasticsearch::instance{ 'es-01': ensure => 'absent' } + " + + apply_manifest(pp, :catch_failures => true) + end + + describe file('/etc/elasticsearch/es-01') do + it { should_not be_directory } + end + + describe package(test_settings['package_name']) do + it { should_not be_installed } + end + + describe service(test_settings['service_name_a']) do + it { should_not be_enabled } + it { should_not be_running } + end + + end + + + describe "install plugin while running ES under user 'elasticsearch'" do + + it 'should run successfully' do + pp = "class { 'elasticsearch': config => { 'node.name' => 'elasticsearch001', 'cluster.name' => '#{test_settings['cluster_name']}' }, manage_repo => true, repo_version => '#{test_settings['repo_version2x']}', java_install => true, elasticsearch_user => 'elasticsearch', elasticsearch_group => 'elasticsearch', version => '2.0.0' } + elasticsearch::instance { 'es-01': config => { 'node.name' => 'elasticsearch001', 'http.port' => '#{test_settings['port_a']}' } } + elasticsearch::plugin{'lmenezes/elasticsearch-kopf': module_dir => 'kopf', instances => 'es-01' } + " + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero + end + + describe service(test_settings['service_name_a']) do + it { should be_enabled } + it { should be_running } + end + + describe package(test_settings['package_name']) do + it { should be_installed } + end + + describe file(test_settings['pid_file_a']) do + it { should be_file } + its(:content) { should match /[0-9]+/ } + end + + it 'make sure the directory exists' do + shell('ls /usr/share/elasticsearch/plugins/kopf/', {:acceptable_exit_codes => 0}) + end + + it 'make sure elasticsearch reports it as existing' do + curl_with_retries('validated plugin as installed', default, "http://localhost:#{test_settings['port_a']}/_nodes/?plugin | grep kopf", 0) + end + + end + + + describe "module removal" do + + it 'should run successfully' do + pp = "class { 'elasticsearch': ensure => 'absent' } + elasticsearch::instance{ 'es-01': ensure => 'absent' } + " + + apply_manifest(pp, :catch_failures => true) + end + + describe file('/etc/elasticsearch/es-01') do + it { should_not be_directory } + end + + describe package(test_settings['package_name']) do + it { should_not be_installed } + end + + describe service(test_settings['service_name_a']) do + it { should_not be_enabled } + it { should_not be_running } + end + + end + + describe 'upgrading', :upgrade => true do + + describe 'Setup 2.0.0' do + it 'should run successful' do + pp = "class { 'elasticsearch': config => { 'node.name' => 'elasticsearch001', 'cluster.name' => '#{test_settings['cluster_name']}' }, manage_repo => true, repo_version => '#{test_settings['repo_version2x']}', java_install => true, version => '2.0.0' } + elasticsearch::instance { 'es-01': config => { 'node.name' => 'elasticsearch001', 'http.port' => '#{test_settings['port_a']}' } } + elasticsearch::plugin{'cloud-aws': instances => 'es-01' } + " + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero + + end + + it 'make sure the directory exists' do + shell('ls /usr/share/elasticsearch/plugins/cloud-aws/', {:acceptable_exit_codes => 0}) + end + + it 'make sure elasticsearch reports it as existing' do + curl_with_retries('validated plugin as installed', default, "http://localhost:#{test_settings['port_a']}/_nodes/?plugin | grep cloud-aws | grep 2.0.0", 0) + end + + end + + describe "Upgrade to 2.0.1" do + it 'Should run succesful' do + pp = "class { 'elasticsearch': config => { 'node.name' => 'elasticsearch001', 'cluster.name' => '#{test_settings['cluster_name']}' }, manage_repo => true, repo_version => '#{test_settings['repo_version2x']}', java_install => true, version => '2.0.1' } + elasticsearch::instance { 'es-01': config => { 'node.name' => 'elasticsearch001', 'http.port' => '#{test_settings['port_a']}' } } + elasticsearch::plugin{'cloud-aws': instances => 'es-01' } + " + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero + + end + + it 'make sure the directory exists' do + shell('ls /usr/share/elasticsearch/plugins/cloud-aws/', {:acceptable_exit_codes => 0}) + end + + it 'make sure elasticsearch reports it as existing' do + curl_with_retries('validated plugin as installed', default, "http://localhost:#{test_settings['port_a']}/_nodes/?plugin | grep cloud-aws | grep 2.0.1", 0) + end + end + + end + + describe "offline install via puppet resource" do + it 'Should run succesful' do + pp = "class { 'elasticsearch': config => { 'node.name' => 'elasticsearch001', 'cluster.name' => '#{test_settings['cluster_name']}' }, manage_repo => true, repo_version => '#{test_settings['repo_version2x']}', java_install => true } + elasticsearch::instance { 'es-01': config => { 'node.name' => 'elasticsearch001', 'http.port' => '#{test_settings['port_a']}' } } + elasticsearch::plugin{'kopf': source => 'puppet:///modules/another/elasticsearch-kopf.zip', instances => 'es-01' } + " + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero + + end + + it 'make sure elasticsearch reports it as existing' do + curl_with_retries('validated plugin as installed', default, "http://localhost:#{test_settings['port_a']}/_nodes/?plugin | grep kopf", 0) + end + + end + + describe "module removal" do + + it 'should run successfully' do + pp = "class { 'elasticsearch': ensure => 'absent' } + elasticsearch::instance{ 'es-01': ensure => 'absent' } + file { '/usr/share/elasticsearch/plugin': + ensure => 'absent', + force => true, + recurse => true, + } + " + + apply_manifest(pp, :catch_failures => true) + end + + describe file('/etc/elasticsearch/es-01') do + it { should_not be_directory } + end + + describe package(test_settings['package_name']) do + it { should_not be_installed } + end + + describe service(test_settings['service_name_a']) do + it { should_not be_enabled } + it { should_not be_running } + end + + end + + describe "offline install via file url" do + it 'Should run succesful' do + pp = "class { 'elasticsearch': config => { 'node.name' => 'elasticsearch001', 'cluster.name' => '#{test_settings['cluster_name']}' }, manage_repo => true, repo_version => '#{test_settings['repo_version2x']}', java_install => true } + elasticsearch::instance { 'es-01': config => { 'node.name' => 'elasticsearch001', 'http.port' => '#{test_settings['port_a']}' } } + elasticsearch::plugin{'kopf': url => 'file:///tmp/elasticsearch-kopf.zip', instances => 'es-01' } + " + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero + + end + + it 'make sure elasticsearch reports it as existing' do + curl_with_retries('validated plugin as installed', default, "http://localhost:#{test_settings['port_a']}/_nodes/?plugin | grep kopf", 0) + end + + end + + describe "module removal" do + + it 'should run successfully' do + pp = "class { 'elasticsearch': ensure => 'absent' } + elasticsearch::instance{ 'es-01': ensure => 'absent' } + " + + apply_manifest(pp, :catch_failures => true) + end + + describe file('/etc/elasticsearch/es-01') do + it { should_not be_directory } + end + + describe package(test_settings['package_name']) do + it { should_not be_installed } + end + + describe service(test_settings['service_name_a']) do + it { should_not be_enabled } + it { should_not be_running } + end + + end + + describe "install via url" do + it 'Should run succesful' do + pp = "class { 'elasticsearch': config => { 'node.name' => 'elasticsearch001', 'cluster.name' => '#{test_settings['cluster_name']}' }, manage_repo => true, repo_version => '#{test_settings['repo_version2x']}', java_install => true } + elasticsearch::instance { 'es-01': config => { 'node.name' => 'elasticsearch001', 'http.port' => '#{test_settings['port_a']}' } } + elasticsearch::plugin{'HQ': url => 'https://github.com/royrusso/elasticsearch-HQ/archive/v2.0.3.zip', instances => 'es-01' } + " + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero + + end + + it 'make sure elasticsearch reports it as existing' do + curl_with_retries('validated plugin as installed', default, "http://localhost:#{test_settings['port_a']}/_nodes/?plugin | grep hq", 0) + end + + end + + describe "module removal" do + + it 'should run successfully' do + pp = "class { 'elasticsearch': ensure => 'absent' } + elasticsearch::instance{ 'es-01': ensure => 'absent' } + " + + apply_manifest(pp, :catch_failures => true) + end + + describe file('/etc/elasticsearch/es-01') do + it { should_not be_directory } + end + + describe package(test_settings['package_name']) do + it { should_not be_installed } + end + + describe service(test_settings['service_name_a']) do + it { should_not be_enabled } + it { should_not be_running } + end + + end + +end diff --git a/elasticsearch/spec/acceptance/xplugins001.rb b/elasticsearch/spec/acceptance/xplugins001.rb new file mode 100644 index 000000000..8618101b3 --- /dev/null +++ b/elasticsearch/spec/acceptance/xplugins001.rb @@ -0,0 +1,91 @@ +require 'spec_helper_acceptance' + +describe "Integration testing" do + + describe "Setup Elasticsearch" do + + it 'should run successfully' do + pp = "class { 'elasticsearch': config => { 'cluster.name' => '#{test_settings['cluster_name']}'}, java_install => true, package_url => '#{test_settings['snapshot_package']}' } + elasticsearch::instance { 'es-01': config => { 'node.name' => 'elasticsearch001', 'http.port' => '#{test_settings['port_a']}' } } + " + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero + end + + + describe service(test_settings['service_name_a']) do + it { should be_enabled } + it { should be_running } + end + + describe package(test_settings['package_name']) do + it { should be_installed } + end + + describe file(test_settings['pid_file_a']) do + it { should be_file } + its(:content) { should match /[0-9]+/ } + end + + describe "Elasticsearch serves requests on" do + it { + curl_with_retries("check ES on #{test_settings['port_a']}", default, "http://localhost:#{test_settings['port_a']}/?pretty=true", 0) + } + end + + describe file('/etc/elasticsearch/es-01/elasticsearch.yml') do + it { should be_file } + it { should contain 'name: elasticsearch001' } + end + + describe file('/usr/share/elasticsearch/templates_import') do + it { should be_directory } + end + + end + + describe "Plugin tests" do + + describe "Install a plugin from official repository" do + + it 'should run successfully' do + pp = "class { 'elasticsearch': config => { 'cluster.name' => '#{test_settings['cluster_name']}'}, java_install => true, package_url => '#{test_settings['snapshot_package']}' } + elasticsearch::instance { 'es-01': config => { 'node.name' => 'elasticsearch001', 'http.port' => '#{test_settings['port_a']}' } } + elasticsearch::plugin{'#{ENV['LICENSE_PLUGIN_NAME']}': instances => 'es-01', url => '#{ENV['LICENSE_PLUGIN_URL']}' } + elasticsearch::plugin{'#{ENV['PLUGIN_NAME']}': instances => 'es-01', url => '#{ENV['PLUGIN_URL']}' } + " + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero + end + + describe service(test_settings['service_name_a']) do + it { should be_enabled } + it { should be_running } + end + + describe package(test_settings['package_name']) do + it { should be_installed } + end + + describe file(test_settings['pid_file_a']) do + it { should be_file } + its(:content) { should match /[0-9]+/ } + end + + it 'make sure the directory exists' do + shell("ls /usr/share/elasticsearch/plugins/#{ENV['PLUGIN_NAME']}", {:acceptable_exit_codes => 0}) + end + + it 'make sure elasticsearch reports it as existing' do + curl_with_retries('validated plugin as installed', default, "http://localhost:#{test_settings['port_a']}/_nodes/?plugin | grep #{ENV['PLUGIN_NAME']}", 0) + end + + end + + end + +end diff --git a/elasticsearch/spec/classes/000_elasticsearch_init_spec.rb b/elasticsearch/spec/classes/000_elasticsearch_init_spec.rb index a0c16492c..019ebc18d 100644 --- a/elasticsearch/spec/classes/000_elasticsearch_init_spec.rb +++ b/elasticsearch/spec/classes/000_elasticsearch_init_spec.rb @@ -62,7 +62,7 @@ it { should contain_file('/usr/share/elasticsearch/scripts') } it { should contain_file('/usr/share/elasticsearch') } it { should contain_file('/usr/share/elasticsearch/lib') } - it { should contain_file('/usr/share/elasticsearch/plugins') } + # it { should contain_file('/usr/share/elasticsearch/plugins') } it { should contain_file('/usr/share/elasticsearch/bin').with(:mode => '0755') } it { should contain_augeas("#{defaults_path}/elasticsearch") } @@ -245,6 +245,7 @@ } it { should contain_package('elasticsearch').with(:ensure => 'purged') } + it { should contain_file('/usr/share/elasticsearch/plugins').with(:ensure => 'absent') } end @@ -293,7 +294,7 @@ it { should contain_file('/etc/elasticsearch').with(:owner => 'myesuser', :group => 'myesgroup') } it { should contain_file('/var/log/elasticsearch').with(:owner => 'myesuser') } it { should contain_file('/usr/share/elasticsearch').with(:owner => 'myesuser', :group => 'myesgroup') } - it { should contain_file('/usr/share/elasticsearch/plugins').with(:owner => 'myesuser', :group => 'myesgroup') } + # it { should contain_file('/usr/share/elasticsearch/plugins').with(:owner => 'myesuser', :group => 'myesgroup') } it { should contain_file('/usr/share/elasticsearch/data').with(:owner => 'myesuser', :group => 'myesgroup') } it { should contain_file('/var/run/elasticsearch').with(:owner => 'myesuser') } if facts[:osfamily] == 'RedHat' end diff --git a/elasticsearch/spec/classes/001_hiera_spec.rb b/elasticsearch/spec/classes/001_hiera_spec.rb index 181907cb6..00025c9af 100644 --- a/elasticsearch/spec/classes/001_hiera_spec.rb +++ b/elasticsearch/spec/classes/001_hiera_spec.rb @@ -119,10 +119,7 @@ } it { should contain_elasticsearch__plugin('mobz/elasticsearch-head/1.0.0').with(:ensure => 'present', :module_dir => 'head', :instances => ['es-01'] ) } - it { should contain_exec('install_plugin_mobz/elasticsearch-head/1.0.0').with(:command => '/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head/1.0.0', :creates => '/usr/share/elasticsearch/plugins/head').that_notifies('Elasticsearch::Service[es-01]') } - it { should contain_file('/usr/share/elasticsearch/plugins/head/.name').with(:content => 'mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('purge_plugin_head_old').with(:onlyif => "test -e /usr/share/elasticsearch/plugins/head && test \"$(cat /usr/share/elasticsearch/plugins/head/.name)\" != 'mobz/elasticsearch-head/1.0.0'", :command => '/usr/share/elasticsearch/bin/plugin --remove head').that_comes_before('Exec[install_plugin_mobz/elasticsearch-head/1.0.0]') } - + it { should contain_elasticsearch_plugin('mobz/elasticsearch-head/1.0.0') } end diff --git a/elasticsearch/spec/classes/005_elasticsearch_repo_spec.rb b/elasticsearch/spec/classes/005_elasticsearch_repo_spec.rb index 9cf543776..1b34da534 100644 --- a/elasticsearch/spec/classes/005_elasticsearch_repo_spec.rb +++ b/elasticsearch/spec/classes/005_elasticsearch_repo_spec.rb @@ -85,7 +85,59 @@ end end - + + context "Override repo key ID" do + + let :params do + default_params.merge({ + :repo_key_id => '46095ACC8548582C1A2699A9D27D666CD88E42B4' + }) + end + + case facts[:osfamily] + when 'Debian' + context 'has override apt key' do + it { is_expected.to contain_apt__source('elasticsearch').with({ + :key => '46095ACC8548582C1A2699A9D27D666CD88E42B4', + })} + end + when 'Suse' + context 'has override yum key' do + it { is_expected.to contain_exec('elasticsearch_suse_import_gpg').with({ + :unless => "test $(rpm -qa gpg-pubkey | grep -i '46095ACC8548582C1A2699A9D27D666CD88E42B4' | wc -l) -eq 1 ", + })} + end + end + + end + + context "Override repo source URL" do + + let :params do + default_params.merge({ + :repo_key_source => 'https://packages.elasticsearch.org/GPG-KEY-elasticsearch' + }) + end + + case facts[:osfamily] + when 'Debian' + context 'has override apt key source' do + it { is_expected.to contain_apt__source('elasticsearch').with({ + :key_source => 'https://packages.elasticsearch.org/GPG-KEY-elasticsearch', + })} + end + when 'RedHat' + context 'has override yum key source' do + it { should contain_yumrepo('elasticsearch').with(:gpgkey => 'https://packages.elasticsearch.org/GPG-KEY-elasticsearch') } + end + when 'Suse' + context 'has override yum key source' do + it { should contain_exec('elasticsearch_suse_import_gpg').with(:command => 'rpmkeys --import https://packages.elasticsearch.org/GPG-KEY-elasticsearch') } + end + end + + end + end end end diff --git a/elasticsearch/spec/defines/004_elasticsearch_plugin_spec.rb b/elasticsearch/spec/defines/004_elasticsearch_plugin_spec.rb index 337362f84..1438d5c56 100644 --- a/elasticsearch/spec/defines/004_elasticsearch_plugin_spec.rb +++ b/elasticsearch/spec/defines/004_elasticsearch_plugin_spec.rb @@ -24,9 +24,7 @@ } end it { should contain_elasticsearch__plugin('mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('install_plugin_mobz/elasticsearch-head/1.0.0').with(:command => '/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head/1.0.0', :creates => '/usr/share/elasticsearch/plugins/head', :notify => 'Elasticsearch::Service[es-01]') } - it { should contain_file('/usr/share/elasticsearch/plugins/head/.name').with(:content => 'mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('purge_plugin_head_old').with(:onlyif => "test -e /usr/share/elasticsearch/plugins/head && test \"$(cat /usr/share/elasticsearch/plugins/head/.name)\" != 'mobz/elasticsearch-head/1.0.0'", :command => '/usr/share/elasticsearch/bin/plugin --remove head', :before => 'Exec[install_plugin_mobz/elasticsearch-head/1.0.0]') } + it { should contain_elasticsearch_plugin('mobz/elasticsearch-head/1.0.0') } end context "Remove a plugin" do @@ -38,71 +36,11 @@ } end it { should contain_elasticsearch__plugin('mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('remove_plugin_mobz/elasticsearch-head/1.0.0').with(:command => '/usr/share/elasticsearch/bin/plugin --remove head', :onlyif => 'test -d /usr/share/elasticsearch/plugins/head', :notify => 'Elasticsearch::Service[es-01]') } + it { should contain_elasticsearch_plugin('mobz/elasticsearch-head/1.0.0').with(:ensure => 'absent') } end end - context 'with auto path' do - - context "Add a plugin" do - - let :params do { - :ensure => 'present', - :instances => 'es-01' - } end - - it { should contain_elasticsearch__plugin('mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('install_plugin_mobz/elasticsearch-head/1.0.0').with(:command => '/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head/1.0.0', :creates => '/usr/share/elasticsearch/plugins/head', :notify => 'Elasticsearch::Service[es-01]') } - it { should contain_file('/usr/share/elasticsearch/plugins/head/.name').with(:content => 'mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('purge_plugin_head_old').with(:onlyif => "test -e /usr/share/elasticsearch/plugins/head && test \"$(cat /usr/share/elasticsearch/plugins/head/.name)\" != 'mobz/elasticsearch-head/1.0.0'", :command => '/usr/share/elasticsearch/bin/plugin --remove head', :before => 'Exec[install_plugin_mobz/elasticsearch-head/1.0.0]') } - end - - context "Remove a plugin" do - - let :params do { - :ensure => 'absent', - :instances => 'es-01' - } end - - it { should contain_elasticsearch__plugin('mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('remove_plugin_mobz/elasticsearch-head/1.0.0').with(:command => '/usr/share/elasticsearch/bin/plugin --remove head', :onlyif => 'test -d /usr/share/elasticsearch/plugins/head', :notify => 'Elasticsearch::Service[es-01]') } - end - - end - - context "Use a proxy" do - - let :params do { - :ensure => 'present', - :module_dir => 'head', - :instances => 'es-01', - :proxy_host => 'my.proxy.com', - :proxy_port => 3128 - } end - - it { should contain_elasticsearch__plugin('mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('install_plugin_mobz/elasticsearch-head/1.0.0').with(:command => '/usr/share/elasticsearch/bin/plugin -DproxyPort=3128 -DproxyHost=my.proxy.com install mobz/elasticsearch-head/1.0.0', :creates => '/usr/share/elasticsearch/plugins/head', :notify => 'Elasticsearch::Service[es-01]') } - it { should contain_file('/usr/share/elasticsearch/plugins/head/.name').with(:content => 'mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('purge_plugin_head_old').with(:onlyif => "test -e /usr/share/elasticsearch/plugins/head && test \"$(cat /usr/share/elasticsearch/plugins/head/.name)\" != 'mobz/elasticsearch-head/1.0.0'", :command => '/usr/share/elasticsearch/bin/plugin --remove head', :before => 'Exec[install_plugin_mobz/elasticsearch-head/1.0.0]') } - - end - - context "Use a proxy from elasticsearch::proxy_url" do - - let(:pre_condition) { 'class {"elasticsearch": config => { "node" => {"name" => "test" }}, proxy_url => "http://localhost:8080/"}'} - - let :params do { - :ensure => 'present', - :module_dir => 'head', - :instances => 'es-01', - } end - - it { should contain_elasticsearch__plugin('mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('install_plugin_mobz/elasticsearch-head/1.0.0').with(:command => '/usr/share/elasticsearch/bin/plugin -DproxyPort=8080 -DproxyHost=localhost install mobz/elasticsearch-head/1.0.0', :creates => '/usr/share/elasticsearch/plugins/head', :notify => 'Elasticsearch::Service[es-01]') } - - end - context 'with url' do context "Add a plugin with full name" do @@ -114,39 +52,7 @@ } end it { should contain_elasticsearch__plugin('mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('install_plugin_mobz/elasticsearch-head/1.0.0').with(:command => '/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head/1.0.0 --url https://github.com/mobz/elasticsearch-head/archive/master.zip', :creates => '/usr/share/elasticsearch/plugins/head', :notify => 'Elasticsearch::Service[es-01]') } - it { should contain_file('/usr/share/elasticsearch/plugins/head/.name').with(:content => 'mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('purge_plugin_head_old').with(:onlyif => "test -e /usr/share/elasticsearch/plugins/head && test \"$(cat /usr/share/elasticsearch/plugins/head/.name)\" != 'mobz/elasticsearch-head/1.0.0'", :command => '/usr/share/elasticsearch/bin/plugin --remove head', :before => 'Exec[install_plugin_mobz/elasticsearch-head/1.0.0]') } - end - - context "Add a plugin with long name and module_dir" do - - let :params do { - :ensure => 'present', - :instances => 'es-01', - :url => 'https://github.com/mobz/elasticsearch-head/archive/master.zip', - :module_dir => 'head' - } end - - it { should contain_elasticsearch__plugin('mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('install_plugin_mobz/elasticsearch-head/1.0.0').with(:command => '/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head/1.0.0 --url https://github.com/mobz/elasticsearch-head/archive/master.zip', :creates => '/usr/share/elasticsearch/plugins/head', :notify => 'Elasticsearch::Service[es-01]') } - it { should contain_file('/usr/share/elasticsearch/plugins/head/.name').with(:content => 'mobz/elasticsearch-head/1.0.0') } - it { should contain_exec('purge_plugin_head_old').with(:onlyif => "test -e /usr/share/elasticsearch/plugins/head && test \"$(cat /usr/share/elasticsearch/plugins/head/.name)\" != 'mobz/elasticsearch-head/1.0.0'", :command => '/usr/share/elasticsearch/bin/plugin --remove head', :before => 'Exec[install_plugin_mobz/elasticsearch-head/1.0.0]') } - end - - context "Add a plugin with short name" do - - let(:title) { 'head' } - let :params do { - :ensure => 'present', - :instances => 'es-01', - :url => 'https://github.com/mobz/elasticsearch-head/archive/master.zip', - } end - - it { should contain_elasticsearch__plugin('head') } - it { should contain_exec('install_plugin_head').with(:command => '/usr/share/elasticsearch/bin/plugin install head --url https://github.com/mobz/elasticsearch-head/archive/master.zip', :creates => '/usr/share/elasticsearch/plugins/head', :notify => 'Elasticsearch::Service[es-01]') } - it { should contain_file('/usr/share/elasticsearch/plugins/head/.name').with(:content => 'head') } - it { should contain_exec('purge_plugin_head_old').with(:onlyif => "test -e /usr/share/elasticsearch/plugins/head && test \"$(cat /usr/share/elasticsearch/plugins/head/.name)\" != 'head'", :command => '/usr/share/elasticsearch/bin/plugin --remove head', :before => 'Exec[install_plugin_head]') } + it { should contain_elasticsearch_plugin('mobz/elasticsearch-head/1.0.0').with(:ensure => 'present', :url => 'https://github.com/mobz/elasticsearch-head/archive/master.zip') } end end @@ -161,11 +67,8 @@ } end it { should contain_elasticsearch__plugin('head') } - it { should contain_file('/tmp/plugin.zip').with(:source => 'puppet:///path/to/my/plugin.zip') } - it { should contain_exec('install_plugin_head').with(:command => '/usr/share/elasticsearch/bin/plugin install head --url file:///tmp/plugin.zip', :creates => '/usr/share/elasticsearch/plugins/head', :notify => 'Elasticsearch::Service[es-01]') } - it { should contain_file('/usr/share/elasticsearch/plugins/head/.name').with(:content => 'head') } - it { should contain_exec('purge_plugin_head_old').with(:onlyif => "test -e /usr/share/elasticsearch/plugins/head && test \"$(cat /usr/share/elasticsearch/plugins/head/.name)\" != 'head'", :command => '/usr/share/elasticsearch/bin/plugin --remove head', :before => 'Exec[install_plugin_head]') } - + it { should contain_file('/opt/elasticsearch/swdl/plugin.zip').with(:source => 'puppet:///path/to/my/plugin.zip') } + it { should contain_elasticsearch_plugin('head').with(:ensure => 'present', :source => '/opt/elasticsearch/swdl/plugin.zip') } end diff --git a/elasticsearch/spec/spec_acceptance_common.rb b/elasticsearch/spec/spec_acceptance_common.rb index eba5f3ac7..5e90f87f4 100644 --- a/elasticsearch/spec/spec_acceptance_common.rb +++ b/elasticsearch/spec/spec_acceptance_common.rb @@ -1,5 +1,6 @@ test_settings['cluster_name'] = SecureRandom.hex(10) + test_settings['repo_version2x'] = '2.x' case fact('osfamily') when 'RedHat' test_settings['repo_version'] = '1.3' diff --git a/elasticsearch/spec/spec_helper_acceptance.rb b/elasticsearch/spec/spec_helper_acceptance.rb index e8802f054..bc1eca8c5 100644 --- a/elasticsearch/spec/spec_helper_acceptance.rb +++ b/elasticsearch/spec/spec_helper_acceptance.rb @@ -86,6 +86,7 @@ def test_settings end scp_to(host, "#{files_dir}/elasticsearch-bigdesk.zip", "/tmp/elasticsearch-bigdesk.zip") + scp_to(host, "#{files_dir}/elasticsearch-kopf.zip", "/tmp/elasticsearch-kopf.zip") end @@ -128,7 +129,12 @@ def test_settings on host, puppet('module', 'install', 'ceritsc-yum'), { :acceptable_exit_codes => [0,1] } end - on(host, 'mkdir -p etc/puppet/modules/another/files/') + if host.is_pe? + on(host, 'sed -i -e "s/PATH=PATH:\/opt\/puppet\/bin:/PATH=PATH:/" ~/.ssh/environment') + end + + on(host, 'mkdir -p etc/puppet/modules/another/files/') + end end diff --git a/elasticsearch/spec/unit/provider/plugin_spec.rb b/elasticsearch/spec/unit/provider/plugin_spec.rb new file mode 100644 index 000000000..79ac15135 --- /dev/null +++ b/elasticsearch/spec/unit/provider/plugin_spec.rb @@ -0,0 +1,112 @@ +require 'spec_helper' + +provider_class = Puppet::Type.type(:elasticsearch_plugin).provider(:plugin) + +describe provider_class do + + let(:resource_name) { 'lmenezes/elasticsearch-kopf' } + let(:resource) do + Puppet::Type.type(:elasticsearch_plugin).new( + :name => resource_name, + :ensure => :present, + :provider => 'plugin' + ) + end + + let(:provider) do + provider = provider_class.new + provider.resource = resource + provider + end + + describe "ES 1.x" do + before(:each) do + provider_class.expects(:es).with('-v').returns("Version: 1.7.1, Build: b88f43f/2015-07-29T09:54:16Z, JVM: 1.7.0_79") + allow(File).to receive(:open) + provider.es_version + end + + let(:shortname) { provider.plugin_name(resource_name) } + + describe 'install' do + it 'installs plugin' do + provider.expects(:plugin).with(['install', [ resource_name] ]) + provider.create + end + + + it 'with url' do + resource[:url] = 'http://url/to/my/plugin.zip' + provider.expects(:plugin).with(['install', [ shortname, '--url', 'http://url/to/my/plugin.zip' ] ]) + provider.create + end + + it 'with local file' do + resource[:source] = '/tmp/plugin.zip' + provider.expects(:plugin).with(['install', [ shortname, '--url', 'file:///tmp/plugin.zip' ] ]) + provider.create + end + + it 'with proxy' do + resource[:proxy_args] = '-dproxyport=3128 -dproxyhost=localhost' + provider.expects(:plugin).with(['-dproxyport=3128 -dproxyhost=localhost', 'install', [resource_name] ]) + provider.create + end + + end + + describe 'removal' do + it 'destroys' do + provider.expects(:plugin).with(['remove', resource_name]) + provider.destroy + end + end + + end + + describe "ES 2.x" do + + before(:each) do + allow(provider_class).to receive(:es).with('-v').and_raise(Puppet::ExecutionFailure) + allow(provider_class).to receive(:es).with('--version').and_return("Version: 2.0.0, Build: de54438/2015-10-22T08:09:48Z, JVM: 1.8.0_66") + allow(File).to receive(:open) + provider.es_version + end + + let(:shortname) { provider.plugin_name(resource_name) } + + describe 'install' do + it 'installs plugin' do + provider.expects(:plugin).with(['install', [ resource_name] ]) + provider.create + end + + it 'with url' do + resource[:url] = 'http://url/to/my/plugin.zip' + provider.expects(:plugin).with(['install', [ 'http://url/to/my/plugin.zip' ] ]) + provider.create + end + + it 'with local file' do + resource[:source] = '/tmp/plugin.zip' + provider.expects(:plugin).with(['install', [ 'file:///tmp/plugin.zip' ] ]) + provider.create + end + + it 'with proxy' do + resource[:proxy_args] = '-dproxyport=3128 -dproxyhost=localhost' + provider.expects(:plugin).with(['-dproxyport=3128 -dproxyhost=localhost', 'install', [resource_name] ]) + provider.create + end + end + + describe 'removal' do + it 'destroys' do + provider.expects(:plugin).with(['remove', resource_name]) + provider.destroy + end + end + + end + +end diff --git a/elasticsearch/spec/unit/type/plugin_spec.rb b/elasticsearch/spec/unit/type/plugin_spec.rb new file mode 100644 index 000000000..cdc6c0856 --- /dev/null +++ b/elasticsearch/spec/unit/type/plugin_spec.rb @@ -0,0 +1,57 @@ +require 'spec_helper' + +describe Puppet::Type.type(:elasticsearch_plugin).provider(:plugin) do + + let(:resource_name) { "lmenezes/elasticsearch-kopf" } + + describe "input validation" do + + let(:type) { Puppet::Type.type(:elasticsearch_plugin) } + + before do + Process.stubs(:euid).returns 0 + Puppet::Util::Storage.stubs(:store) + end + + it "should default to being installed" do + plugin = Puppet::Type.type(:elasticsearch_plugin).new(:name => resource_name ) + expect(plugin.should(:ensure)).to eq(:present) + end + + describe "when validating attributes" do + [:name, :source, :url, :proxy_args].each do |param| + it "should have a #{param} parameter" do + expect(type.attrtype(param)).to eq(:param) + end + end + + it "should have an ensure property" do + expect(type.attrtype(:ensure)).to eq(:property) + end + end + + end + +end + + describe 'other tests' do + + prov_c = Puppet::Type.type(:elasticsearch_plugin).provider(:plugin) + + describe prov_c do + + it 'should install a plugin' do + resource = Puppet::Type.type(:elasticsearch_plugin).new( + :name => "lmenezes/elasticsearch-kopf", + :ensure => :present + ) + allow(File).to receive(:open) + provider = prov_c.new(resource) + provider.expects(:es).with('-v').returns('Version: 1.7.3, Build: b88f43f/2015-07-29T09:54:16Z, JVM: 1.7.0_79') + provider.expects(:plugin).with(['install', ['lmenezes/elasticsearch-kopf']]) + provider.create + end + + end + end + diff --git a/elasticsearch/templates/etc/init.d/elasticsearch.Debian.erb b/elasticsearch/templates/etc/init.d/elasticsearch.Debian.erb index 7138802e6..cccc8dbd6 100644 --- a/elasticsearch/templates/etc/init.d/elasticsearch.Debian.erb +++ b/elasticsearch/templates/etc/init.d/elasticsearch.Debian.erb @@ -111,6 +111,7 @@ export ES_HEAP_NEWSIZE export ES_DIRECT_SIZE export ES_JAVA_OPTS export ES_CLASSPATH +export ES_INCLUDE # Check DAEMON exists test -x $DAEMON || exit 0 diff --git a/elasticsearch/templates/etc/init.d/elasticsearch.RedHat.erb b/elasticsearch/templates/etc/init.d/elasticsearch.RedHat.erb index 2e7f0cf5b..3cab1f57f 100644 --- a/elasticsearch/templates/etc/init.d/elasticsearch.RedHat.erb +++ b/elasticsearch/templates/etc/init.d/elasticsearch.RedHat.erb @@ -43,6 +43,7 @@ export ES_DIRECT_SIZE export ES_JAVA_OPTS export ES_CLASSPATH export JAVA_HOME +export ES_INCLUDE lockfile=/var/lock/subsys/$prog diff --git a/midonet/.gitreview b/midonet/.gitreview index 0aaf6104e..e3a441674 100644 --- a/midonet/.gitreview +++ b/midonet/.gitreview @@ -2,4 +2,3 @@ host=review.gerrithub.io port=29418 project=midonet/puppet-midonet.git -defaultbranch=stable/v2015.06 diff --git a/midonet/README.md b/midonet/README.md index 8a20d8ffe..7b47843e2 100644 --- a/midonet/README.md +++ b/midonet/README.md @@ -22,8 +22,8 @@ Puppet module for install MidoNet components. MidoNet is an Apache licensed production grade network virtualization software for Infrastructure-as-a-Service (IaaS) clouds. This module provides the puppet -manifests to install all the components to deploy easily MidoNet in a production -environment. +manifests to install all the components to deploy easily MidoNet in a +production environment. To know all the components and how they relate each other, check out [midonet reference architecture @@ -52,11 +52,11 @@ be useful. ## Usage To use this module in a more advanced way, please check out the -[reference](#reference) section of this document. It is worth to highlight that all -the input variables have already a default input value, in a yaml document. +[reference](#reference) section of this document. It is worth to highlight that +all the input variables have already a default input value, in a yaml document. (We use R.I.Piennar [module data](https://www.devco.net/archives/2013/12/08/better-puppet-modules-using-hiera-data.php)) -To leverage this feature, please add the following in your `/etc/hiera.yaml` (or -the Hiera configuration file that you are using): +To leverage this feature, please add the following in your `/etc/puppet/hiera.yaml` +(or the Hiera configuration file that you are using): --- :backends: diff --git a/midonet/metadata.json b/midonet/metadata.json index 2da22a95c..6701551a5 100644 --- a/midonet/metadata.json +++ b/midonet/metadata.json @@ -1,6 +1,6 @@ { "name": "midonet-midonet", - "version": "2015.6.7", + "version": "2015.6.0", "author": "MidoNet", "summary": "Configure and install MidoNet components", "license": "Apache-2.0", @@ -14,7 +14,8 @@ { "name":"midonet-cassandra","version_requirement":">=1.0.0" }, { "name":"puppetlabs-apt","version_requirement":">=1.7.0 <2.0.0" }, { "name":"puppetlabs-java","version_requirement":">=1.3.0" }, - { "name":"puppetlabs-tomcat","version_requirement":">=1.2.0" } + { "name":"puppetlabs-tomcat","version_requirement":">=1.2.0" }, + { "name":"puppetlabs-stdlib","version_requirement":">= 4.2.0 < 5.0.0" } ], "tags": ["openstack", "sdn", "midonet"], "operatingsystem_support": [ diff --git a/midonet/test/init.pp b/midonet/test/init.pp deleted file mode 100644 index 3a01941c0..000000000 --- a/midonet/test/init.pp +++ /dev/null @@ -1,19 +0,0 @@ -# The baseline for module testing used by Puppet Labs is that each manifest -# should have a corresponding test manifest that declares that class or defined -# type. -# -# Tests are then run by using puppet apply --noop (to check for compilation -# errors and view a log of events) or by fully applying the test in a virtual -# environment (to compare the resulting system state to the desired state). -# -# Learn more about module testing here: -# http://docs.puppetlabs.com/guides/tests_smoke.html -# - -# Fake the facter when it compiles. The augeas version that will be installed -# will be this one -if empty($::augeasversion) { - $augeasversion = '1.0.0' -} - -class {'midonet':} diff --git a/midonet/test/integration/default/bats/verify_server.bats b/midonet/test/integration/default/bats/verify_server.bats deleted file mode 100644 index adefa79ec..000000000 --- a/midonet/test/integration/default/bats/verify_server.bats +++ /dev/null @@ -1,160 +0,0 @@ -# Test verify methods for midonet_repository - -command_exists() { - command -v "$@" > /dev/null 2>&1 -} - -# Code copied unashamedly from http://get.docker.io -get_distro() { - lsb_dist='' - if command_exists lsb_release; then - lsb_dist="$(lsb_release -si)" - fi - if [ -z "$lsb_dist" ] && [ -r /etc/lsb-release ]; then - lsb_dist="$(. /etc/lsb-release && echo "$DISTRIB_ID")" - fi - if [ -z "$lsb_dist" ] && [ -r /etc/debian_version ]; then - lsb_dist='debian' - fi - if [ -z "$lsb_dist" ] && [ -r /etc/redhat-release ]; then - lsb_dist='red-hat' - fi - if [ -z "$lsb_dist" ] && [ -r /etc/os-release ]; then - lsb_dist="$(. /etc/os-release && echo "$ID")" - fi - - distro=$(echo "$lsb_dist" | tr '[:upper:]' '[:lower:]') -} - -get_distro - -@test 'midonet repo is set' { - - case $distro in - ubuntu) - run file /etc/apt/sources.list.d/midonet.list - [ "$status" -eq 0 ] - run file /etc/apt/sources.list.d/midonet-openstack-integration.list - [ "$status" -eq 0 ] - ;; - centos|red-hat) - run ls /etc/yum.repos.d/midonet.repo - [ "$status" -eq 0 ] - run ls /etc/yum.repos.d/midonet-openstack-integration.repo - [ "$status" -eq 0 ] - ;; - *) - exit 1; - esac -} - -@test 'midonet packages are available' { - case $distro in - ubuntu) - run bash -c "apt-cache search mido | grep midolman" - [ "$status" -eq 0 ] - run bash -c "apt-cache search mido | grep midonet-api" - [ "$status" -eq 0 ] - run bash -c "apt-cache search mido | grep python-midonetclient" - [ "$status" -eq 0 ] - run bash -c "apt-cache search mido | grep python-neutron-plugin-midonet" - [ "$status" -eq 0 ] - run bash -c "apt-cache search dsc20" - [ "$status" -eq 0 ] - ;; - centos|red-hat) - run bash -c "yum search mido | grep midolman" - [ "$status" -eq 0 ] - run bash -c "yum search mido | grep midonet-api" - [ "$status" -eq 0 ] - run bash -c "yum search mido | grep python-midonetclient" - [ "$status" -eq 0 ] - run bash -c "yum search mido | grep python-neutron-plugin-midonet" - [ "$status" -eq 0 ] - run bash -c "yum search dsc20-2.0.10-1" - [ "$status" -eq 0 ] - ;; - *) - exit 1; - esac -} - -@test 'zookeeper is running' { - case $distro in - ubuntu) - run bash -c "sudo /usr/share/zookeeper/bin/zkServer.sh status || sudo /usr/sbin/zkServer.sh status" - [ "$status" -eq 0 ] - ;; - centos|red-hat) - run sudo /usr/sbin/zkServer.sh status - [ "$status" -eq 0 ] - ;; - *) - exit 1; - esac -} - -@test 'cassandra is running' { - case $distro in - ubuntu) - run sudo service cassandra status - [ "$status" -eq 0 ] - ;; - centos|red-hat) - run sudo service cassandra status - [ "$status" -eq 0 ] - ;; - *) - exit 1; - esac -} - -@test 'midonet-agent is running' { - case $distro in - ubuntu) - run sudo service midolman status - [ "$status" -eq 0 ] - ;; - centos|red-hat) - run sudo service midolman status - [ "$status" -eq 0 ] - ;; - *) - exit 1; - esac -} - -@test 'midonet-api is running' { - case $distro in - ubuntu) - run sudo service tomcat7 status - [ "$status" -eq 0 ] - ;; - centos|red-hat) - run sudo service tomcat status - [ "$status" -eq 0 ] - ;; - *) - exit 1; - esac -} - -@test 'midonet-cli is installed' { - case $distro in - ubuntu) - run bash -c "dpkg -l | grep python-midonetclient" - [ "$status" -eq 0 ] - ;; - centos|red-hat) - run bash -c "rpm -qa | grep python-midonetclient" - [ "$status" -eq 0 ] - ;; - *) - exit 1; - esac -} - -@test 'midonetrc file is created' { - run bash -c "file /root/.midonetrc" - [ "$status" -eq 0 ] -} diff --git a/opendaylight/CONTRIBUTING.markdown b/opendaylight/CONTRIBUTING.markdown index 7da349105..085948f69 100644 --- a/opendaylight/CONTRIBUTING.markdown +++ b/opendaylight/CONTRIBUTING.markdown @@ -101,7 +101,8 @@ The testing tools have a number of dependencies. We use [Bundler][10] to make installing them easy. ``` -[~/puppet-opendaylight]$ sudo yum install -y rubygems ruby-devel gcc-c++ zlib-devel patch +[~/puppet-opendaylight]$ sudo yum install -y rubygems ruby-devel gcc-c++ zlib-devel \ + patch redhat-rpm-config make [~/puppet-opendaylight]$ gem install bundler [~/puppet-opendaylight]$ bundle install [~/puppet-opendaylight]$ bundle update diff --git a/opendaylight/Rakefile b/opendaylight/Rakefile index 6cbc73988..a36f79c39 100644 --- a/opendaylight/Rakefile +++ b/opendaylight/Rakefile @@ -57,6 +57,9 @@ task :centos_tarball do sh "RS_SET=centos-7 INSTALL_METHOD=tarball bundle exec rake beaker" end +# NB: The centos:7.0.1406 and centos:7.1.1503 tags have fakesytemd, not +# the actually-functional systemd-container installed on centos:7 +# https://github.com/CentOS/sig-cloud-instance-build/commit/3bf1e7bbf14deaa8c047c1dfbead6d0e8d0665f2 desc "Run Beaker tests against CentOS 7 Docker node." task :centos_7_docker do sh "RS_SET=centos-7-docker INSTALL_METHOD=rpm bundle exec rake beaker" @@ -82,19 +85,17 @@ task :ubuntu_1404 do sh "RS_SET=ubuntu-1404 INSTALL_METHOD=tarball bundle exec rake beaker" end -desc "Run Beaker tests against Ubuntu 15.04 node." -task :ubuntu_1504 do - sh "RS_SET=ubuntu-1504 INSTALL_METHOD=tarball bundle exec rake beaker" +desc "Run Beaker tests against Ubuntu 14.04 Docker node." +task :ubuntu_1404_docker do + sh "RS_SET=ubuntu-1404-docker INSTALL_METHOD=tarball bundle exec rake beaker" end +# Note: Puppet currently doesn't support Ubuntu versions newer than 14.04 +# https://docs.puppetlabs.com/guides/install_puppet/install_debian_ubuntu.html + desc "All tests, including Beaker tests against all nodes." task :acceptance => [ :test, - :centos, - :centos_tarball, - :fedora_20, - :fedora_21, - :fedora_22, - :ubuntu_1404, - :ubuntu_1504, + :centos_7_docker, + :ubuntu_1404_docker, ] diff --git a/opendaylight/Vagrantfile b/opendaylight/Vagrantfile new file mode 100644 index 000000000..015762ffc --- /dev/null +++ b/opendaylight/Vagrantfile @@ -0,0 +1,47 @@ +# -*- mode: ruby -*- +# vi: set ft=ruby : + +Vagrant.configure(2) do |config| + + # Not doing this causes `puppet apply` to fail at catalog compile + config.vm.synced_folder ".", "/home/vagrant/puppet-opendaylight", type: "rsync" + config.vm.synced_folder ".", "/vagrant", disabled: true + + # We run out of RAM once ODL starts with default 500MB + config.vm.provider :libvirt do |libvirt| + libvirt.memory = 4096 + libvirt.cpus = 2 + end + + config.vm.define "f23" do |f23| + f23.vm.box = "fedora/23-cloud-base" + + f23.vm.provision "shell", inline: "dnf update -y" + + # Install required gems via Bundler + f23.vm.provision "shell", inline: "dnf install -y rubygems ruby-devel gcc-c++ zlib-devel patch redhat-rpm-config make" + f23.vm.provision "shell", inline: "gem install bundler" + f23.vm.provision "shell", inline: "echo export PATH=$PATH:/usr/local/bin >> /home/vagrant/.bashrc" + f23.vm.provision "shell", inline: "echo export PATH=$PATH:/usr/local/bin >> /root/.bashrc" + f23.vm.provision "shell", inline: 'su -c "cd /home/vagrant/puppet-opendaylight; bundle install" vagrant' + f23.vm.provision "shell", inline: 'su -c "cd /home/vagrant/puppet-opendaylight; bundle update" vagrant' + + # Git is required for cloning Puppet module deps in `rake test` + f23.vm.provision "shell", inline: "dnf install -y git" + + # Install Docker for Docker-based Beaker tests + f23.vm.provision "shell", inline: "tee /etc/yum.repos.d/docker.repo <<-'EOF' +[dockerrepo] +name=Docker Repository +baseurl=https://yum.dockerproject.org/repo/main/fedora/$releasever/ +enabled=1 +gpgcheck=1 +gpgkey=https://yum.dockerproject.org/gpg +EOF +" + f23.vm.provision "shell", inline: "dnf install -y docker-engine xfsprogs" + f23.vm.provision "shell", inline: "usermod -a -G docker vagrant" + f23.vm.provision "shell", inline: "systemctl start docker" + f23.vm.provision "shell", inline: "systemctl enable docker" + end +end diff --git a/opendaylight/spec/acceptance/nodesets/ubuntu-1404-docker.yml b/opendaylight/spec/acceptance/nodesets/ubuntu-1404-docker.yml new file mode 100644 index 000000000..9b1c3d4bf --- /dev/null +++ b/opendaylight/spec/acceptance/nodesets/ubuntu-1404-docker.yml @@ -0,0 +1,7 @@ +HOSTS: + ubuntu-1404-docker: + platform: ubuntu-14.04-x64 + image: ubuntu:14.04 + hypervisor: docker +CONFIG: + type: foss diff --git a/opendaylight/spec/acceptance/nodesets/ubuntu-1504.yml b/opendaylight/spec/acceptance/nodesets/ubuntu-1504.yml deleted file mode 100644 index 25bee91ef..000000000 --- a/opendaylight/spec/acceptance/nodesets/ubuntu-1504.yml +++ /dev/null @@ -1,12 +0,0 @@ -HOSTS: - ubuntu-1504: - roles: - - master - platform: ubuntu-1504-amd64 - box: boxcutter/ubuntu1504 - box_url: https://atlas.hashicorp.com/boxcutter/boxes/ubuntu1504 - hypervisor: vagrant - -CONFIG: - log_level: verbose - type: foss diff --git a/opendaylight/spec/spec_helper_acceptance.rb b/opendaylight/spec/spec_helper_acceptance.rb index f7f64a04d..69e297f7d 100644 --- a/opendaylight/spec/spec_helper_acceptance.rb +++ b/opendaylight/spec/spec_helper_acceptance.rb @@ -158,7 +158,7 @@ def generic_validations() describe package('java-1.8.0-openjdk') do it { should be_installed } end - elsif ENV['RS_SET'] == 'ubuntu-1404' + elsif ['ubuntu-1404', 'ubuntu-1404-docker'].include? ENV['RS_SET'] # Ubuntu-specific validations # Verify ODL Upstart config file diff --git a/openstacklib/manifests/openstackclient.pp b/openstacklib/manifests/openstackclient.pp index 096741c22..25e23adcd 100644 --- a/openstacklib/manifests/openstackclient.pp +++ b/openstacklib/manifests/openstackclient.pp @@ -11,8 +11,5 @@ class openstacklib::openstackclient( $package_ensure = 'present', ){ - package { 'python-openstackclient': - ensure => $package_ensure, - tag => 'openstack', - } + ensure_packages('python-openstackclient', {'ensure' => $package_ensure, tag => 'openstack'}) } diff --git a/openstacklib/spec/defines/openstacklib_db_postgresql_spec.rb b/openstacklib/spec/defines/openstacklib_db_postgresql_spec.rb index 4e2dfc7c7..6a9f3ce26 100644 --- a/openstacklib/spec/defines/openstacklib_db_postgresql_spec.rb +++ b/openstacklib/spec/defines/openstacklib_db_postgresql_spec.rb @@ -9,11 +9,18 @@ { :password_hash => password_hash } end + let (:pre_condition) do + "include ::postgresql::server" + end + context 'on a RedHat osfamily' do let :facts do { - :postgres_default_version => '8.4', - :osfamily => 'RedHat' + :osfamily => 'RedHat', + :operatingsystem => 'RedHat', + :operatingsystemrelease => '7.1', + :operatingsystemmajrelease => '7', + :concat_basedir => '/tmp', } end @@ -45,7 +52,8 @@ context 'when notifying other resources' do let :pre_condition do - 'exec { "nova-db-sync": }' + "include ::postgresql::server + exec { 'nova-db-sync': }" end let :params do { :notify => 'Exec[nova-db-sync]'}.merge(required_params) @@ -56,7 +64,8 @@ context 'when required for other openstack services' do let :pre_condition do - 'service {"keystone":}' + "include ::postgresql::server + service {'keystone':}" end let :title do 'keystone' @@ -73,7 +82,11 @@ context 'on a Debian osfamily' do let :facts do { - :osfamily => 'Debian' + :osfamily => 'Debian', + :operatingsystem => 'Debian', + :operatingsystemrelease => 'jessie', + :operatingsystemmajrelease => '8.2', + :concat_basedir => '/tmp', } end @@ -105,7 +118,8 @@ context 'when notifying other resources' do let :pre_condition do - 'exec { "nova-db-sync": }' + "include ::postgresql::server + exec { 'nova-db-sync': }" end let :params do { :notify => 'Exec[nova-db-sync]'}.merge(required_params) @@ -116,7 +130,8 @@ context 'when required for other openstack services' do let :pre_condition do - 'service {"keystone":}' + "include ::postgresql::server + service {'keystone':}" end let :title do 'keystone' diff --git a/rabbitmq/lib/puppet/provider/rabbitmq_erlang_cookie/ruby.rb b/rabbitmq/lib/puppet/provider/rabbitmq_erlang_cookie/ruby.rb index 74a96168f..2784c3888 100644 --- a/rabbitmq/lib/puppet/provider/rabbitmq_erlang_cookie/ruby.rb +++ b/rabbitmq/lib/puppet/provider/rabbitmq_erlang_cookie/ruby.rb @@ -3,10 +3,15 @@ Puppet::Type.type(:rabbitmq_erlang_cookie).provide(:ruby) do defaultfor :feature => :posix - has_command(:puppet, 'puppet') do - environment :PATH => '/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin' + + env_path = '/opt/puppetlabs/bin:/usr/local/bin:/usr/bin:/bin' + puppet_path = Puppet::Util.withenv(:PATH => env_path) do + Puppet::Util.which('puppet') end + confine :false => puppet_path.nil? + has_command(:puppet, puppet_path) unless puppet_path.nil? + def exists? # Hack to prevent the create method from being called. # We never need to create or destroy this resource, only change its value diff --git a/rabbitmq/manifests/init.pp b/rabbitmq/manifests/init.pp index f1346bbc8..2bde1ec98 100644 --- a/rabbitmq/manifests/init.pp +++ b/rabbitmq/manifests/init.pp @@ -103,7 +103,9 @@ } validate_bool($wipe_db_on_cookie_change) validate_bool($tcp_keepalive) - validate_re($file_limit, '^(\d+|-1|unlimited|infinity)$', '$file_limit must be a positive integer, \'-1\', \'unlimited\', or \'infinity\'.') + # using sprintf for conversion to string, because "${file_limit}" doesn't + # pass lint, despite being nicer + validate_re(sprintf('%s', $file_limit), '^(\d+|-1|unlimited|infinity)$', '$file_limit must be a positive integer, \'-1\', \'unlimited\', or \'infinity\'.') # Validate service parameters. validate_re($service_ensure, '^(running|stopped)$') validate_bool($service_manage) diff --git a/rabbitmq/manifests/params.pp b/rabbitmq/manifests/params.pp index 74b2a7d41..1101ebb36 100644 --- a/rabbitmq/manifests/params.pp +++ b/rabbitmq/manifests/params.pp @@ -40,7 +40,7 @@ $package_ensure = 'installed' $package_name = 'rabbitmq-server' $service_name = 'rabbitmq-server' - $package_provider = 'rpm' + $package_provider = 'yum' $version = '3.1.5-1' $rabbitmq_user = 'rabbitmq' $rabbitmq_group = 'rabbitmq' diff --git a/rabbitmq/spec/classes/rabbitmq_spec.rb b/rabbitmq/spec/classes/rabbitmq_spec.rb index 45b4b7cd2..c65b44e8f 100644 --- a/rabbitmq/spec/classes/rabbitmq_spec.rb +++ b/rabbitmq/spec/classes/rabbitmq_spec.rb @@ -160,6 +160,11 @@ it { should contain_file('/etc/default/rabbitmq-server').with_content(/ulimit -n 1234/) } end + context 'with file_limit => 1234' do + let(:params) {{ :file_limit => 1234 }} + it { should contain_file('/etc/default/rabbitmq-server').with_content(/ulimit -n 1234/) } + end + context 'with file_limit => \'-42\'' do let(:params) {{ :file_limit => '-42' }} it 'does not compile' do @@ -176,7 +181,7 @@ end context 'on Redhat' do - let(:facts) {{ :osfamily => 'RedHat' }} + let(:facts) {{ :osfamily => 'RedHat', :operatingsystemmajrelease => '7' }} it 'includes rabbitmq::repo::rhel' do should contain_class('rabbitmq::repo::rhel') should contain_exec('rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc') @@ -251,7 +256,7 @@ context 'on Redhat' do let(:params) {{ :repos_ensure => false }} - let(:facts) {{ :osfamily => 'RedHat' }} + let(:facts) {{ :osfamily => 'RedHat', :operatingsystemmajrelease => '7' }} it 'does not import repo public key when repos_ensure is false' do should contain_class('rabbitmq::repo::rhel') should_not contain_exec('rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc') @@ -260,7 +265,7 @@ context 'on Redhat' do let(:params) {{ :repos_ensure => true }} - let(:facts) {{ :osfamily => 'RedHat' }} + let(:facts) {{ :osfamily => 'RedHat', :operatingsystemmajrelease => '7' }} it 'does import repo public key when repos_ensure is true' do should contain_class('rabbitmq::repo::rhel') should contain_exec('rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc') @@ -269,7 +274,7 @@ context 'on Redhat' do let(:params) {{ :manage_repos => false }} - let(:facts) {{ :osfamily => 'RedHat' }} + let(:facts) {{ :osfamily => 'RedHat', :operatingsystemmajrelease => '7' }} it 'does not import repo public key when manage_repos is false' do should_not contain_class('rabbitmq::repo::rhel') should_not contain_exec('rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc') @@ -278,7 +283,7 @@ context 'on Redhat' do let(:params) {{ :manage_repos => true }} - let(:facts) {{ :osfamily => 'RedHat' }} + let(:facts) {{ :osfamily => 'RedHat', :operatingsystemmajrelease => '7' }} it 'does import repo public key when manage_repos is true' do should contain_class('rabbitmq::repo::rhel') should contain_exec('rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc') @@ -287,7 +292,7 @@ context 'on Redhat' do let(:params) {{ :manage_repos => false, :repos_ensure => true }} - let(:facts) {{ :osfamily => 'RedHat' }} + let(:facts) {{ :osfamily => 'RedHat', :operatingsystemmajrelease => '7' }} it 'does not import repo public key when manage_repos is false and repos_ensure is true' do should_not contain_class('rabbitmq::repo::rhel') should_not contain_exec('rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc') @@ -296,7 +301,7 @@ context 'on Redhat' do let(:params) {{ :manage_repos => true, :repos_ensure => true }} - let(:facts) {{ :osfamily => 'RedHat' }} + let(:facts) {{ :osfamily => 'RedHat', :operatingsystemmajrelease => '7' }} it 'does import repo public key when manage_repos is true and repos_ensure is true' do should contain_class('rabbitmq::repo::rhel') should contain_exec('rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc') @@ -305,7 +310,7 @@ context 'on Redhat' do let(:params) {{ :manage_repos => false, :repos_ensure => false }} - let(:facts) {{ :osfamily => 'RedHat' }} + let(:facts) {{ :osfamily => 'RedHat', :operatingsystemmajrelease => '7' }} it 'does not import repo public key when manage_repos is false and repos_ensure is false' do should_not contain_class('rabbitmq::repo::rhel') should_not contain_exec('rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc') @@ -314,7 +319,7 @@ context 'on Redhat' do let(:params) {{ :manage_repos => true, :repos_ensure => false }} - let(:facts) {{ :osfamily => 'RedHat' }} + let(:facts) {{ :osfamily => 'RedHat', :operatingsystemmajrelease => '7' }} it 'does not import repo public key when manage_repos is true and repos_ensure is false' do should contain_class('rabbitmq::repo::rhel') should_not contain_exec('rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc') @@ -391,12 +396,24 @@ end ['Debian', 'RedHat', 'SUSE', 'Archlinux'].each do |distro| - context "on #{distro}" do - let(:facts) {{ - :osfamily => distro, + osfacts = { + :osfamily => distro, + } + + case distro + when 'Debian' + osfacts.merge!({ :lsbdistcodename => 'squeeze', :lsbdistid => 'Debian' - }} + }) + when 'RedHat' + osfacts.merge!({ + :operatingsystemmajrelease => '7', + }) + end + + context "on #{distro}" do + let(:facts) { osfacts } it { should contain_class('rabbitmq::install') } it { should contain_class('rabbitmq::config') } @@ -447,7 +464,6 @@ end context 'configures config_cluster' do - let(:facts) {{ :osfamily => distro, :lsbdistid => 'Debian' }} let(:params) {{ :config_cluster => true, :cluster_nodes => ['hare-1', 'hare-2'], @@ -1111,13 +1127,13 @@ ## rabbitmq::install ## context "on RHEL" do - let(:facts) {{ :osfamily => 'RedHat' }} + let(:facts) {{ :osfamily => 'RedHat', :operatingsystemmajrelease => '7' }} let(:params) {{ :package_source => 'http://www.rabbitmq.com/releases/rabbitmq-server/v3.2.3/rabbitmq-server-3.2.3-1.noarch.rpm' }} it 'installs the rabbitmq package' do should contain_package('rabbitmq-server').with( 'ensure' => 'installed', 'name' => 'rabbitmq-server', - 'provider' => 'rpm', + 'provider' => 'yum', 'source' => 'http://www.rabbitmq.com/releases/rabbitmq-server/v3.2.3/rabbitmq-server-3.2.3-1.noarch.rpm' ) end @@ -1182,9 +1198,25 @@ end ['RedHat', 'SuSE'].each do |distro| + osfacts = { + :osfamily => distro, + } + + case distro + when 'Debian' + osfacts.merge!({ + :lsbdistcodename => 'squeeze', + :lsbdistid => 'Debian' + }) + when 'RedHat' + osfacts.merge!({ + :operatingsystemmajrelease => '7', + }) + end + describe "repo management on #{distro}" do describe 'imports the key' do - let(:facts) {{ :osfamily => distro }} + let(:facts) { osfacts } let(:params) {{ :package_gpg_key => 'http://www.rabbitmq.com/rabbitmq-signing-key-public.asc' }} it { should contain_exec("rpm --import #{params[:package_gpg_key]}").with( diff --git a/redis/.gitignore b/redis/.gitignore index c6c26c3b8..7a0324898 100644 --- a/redis/.gitignore +++ b/redis/.gitignore @@ -1,7 +1,9 @@ *.swp .DS_Store +.vagrant/ +.yardoc Gemfile.lock +doc +log/ pkg spec/fixtures -.yardoc -doc diff --git a/redis/Gemfile b/redis/Gemfile index a8f3419df..38f9229d7 100644 --- a/redis/Gemfile +++ b/redis/Gemfile @@ -1,5 +1,12 @@ source 'https://rubygems.org' +# special dependencies for Ruby 1.8 +# since there are still several OSes with it +if RUBY_VERSION =~ /^1\.8\./ + gem 'rspec-core', '~> 3.1.7' + gem 'nokogiri', '~> 1.5.0' +end + puppetversion = ENV.key?('PUPPET_VERSION') ? "~> #{ENV['PUPPET_VERSION']}" : ['>= 3.2.1'] gem 'puppet', puppetversion @@ -18,3 +25,6 @@ gem 'puppet-lint', '>=0.3.2' gem 'rspec-puppet', '>=0.1.6' gem 'puppetlabs_spec_helper', '>=0.4.1' +gem 'beaker-rspec' +gem 'bundler', '<= 1.10.5' +gem 'vagrant-wrapper' diff --git a/redis/README.md b/redis/README.md index 6adfef6ec..9c4e12e3f 100644 --- a/redis/README.md +++ b/redis/README.md @@ -26,6 +26,16 @@ #masterauth => 'secret'; } +### Redis 3.0 Clustering + + class { 'redis': + bind => '10.0.1.2', + appendonly => true, + cluster_enabled => true, + cluster_config_file => 'nodes.conf', + cluster_node_timeout => 5000, + } + ### Manage repositories Disabled by default but if you really want the module to manage the required diff --git a/redis/manifests/config.pp b/redis/manifests/config.pp index c7f5324ad..b0ff449b9 100644 --- a/redis/manifests/config.pp +++ b/redis/manifests/config.pp @@ -3,47 +3,52 @@ # This class provides configuration for Redis. # class redis::config { - $daemonize = $::redis::daemonize - $pid_file = $::redis::pid_file - $port = $::redis::port + $activerehashing = $::redis::activerehashing + $appendfsync = $::redis::appendfsync + $appendonly = $::redis::appendonly + $auto_aof_rewrite_min_size = $::redis::auto_aof_rewrite_min_size + $auto_aof_rewrite_percentage = $::redis::auto_aof_rewrite_percentage $bind = $::redis::bind - $timeout = $::redis::timeout - $log_level = $::redis::log_level - $log_file = $::redis::log_file - $syslog_enabled = $::redis::syslog_enabled - $syslog_facility = $::redis::syslog_facility + $cluster_config_file = $::redis::cluster_config_file + $cluster_enabled = $::redis::cluster_enabled + $cluster_node_timeout = $::redis::cluster_node_timeout + $daemonize = $::redis::daemonize $databases = $::redis::databases - $rdbcompression = $::redis::rdbcompression $dbfilename = $::redis::dbfilename - $workdir = $::redis::workdir - $slaveof = $::redis::slaveof + $extra_config_file = $::redis::extra_config_file + $hash_max_ziplist_entries = $::redis::hash_max_ziplist_entries + $hash_max_ziplist_value = $::redis::hash_max_ziplist_value + $hz = $::redis::hz + $list_max_ziplist_entries = $::redis::list_max_ziplist_entries + $list_max_ziplist_value = $::redis::list_max_ziplist_value + $log_file = $::redis::log_file + $log_level = $::redis::log_level $masterauth = $::redis::masterauth - $slave_serve_stale_data = $::redis::slave_serve_stale_data - $slave_read_only = $::redis::slave_read_only - $repl_timeout = $::redis::repl_timeout - $requirepass = $::redis::requirepass - $save_db_to_disk = $::redis::save_db_to_disk $maxclients = $::redis::maxclients $maxmemory = $::redis::maxmemory $maxmemory_policy = $::redis::maxmemory_policy $maxmemory_samples = $::redis::maxmemory_samples - $appendonly = $::redis::appendonly - $appendfsync = $::redis::appendfsync $no_appendfsync_on_rewrite = $::redis::no_appendfsync_on_rewrite - $auto_aof_rewrite_percentage = $::redis::auto_aof_rewrite_percentage - $auto_aof_rewrite_min_size = $::redis::auto_aof_rewrite_min_size + $pid_file = $::redis::pid_file + $port = $::redis::port + $rdbcompression = $::redis::rdbcompression + $repl_timeout = $::redis::repl_timeout + $requirepass = $::redis::requirepass + $save_db_to_disk = $::redis::save_db_to_disk + $set_max_intset_entries = $::redis::set_max_intset_entries + $slave_read_only = $::redis::slave_read_only + $slave_serve_stale_data = $::redis::slave_serve_stale_data + $slaveof = $::redis::slaveof $slowlog_log_slower_than = $::redis::slowlog_log_slower_than $slowlog_max_len = $::redis::slowlog_max_len - $hash_max_ziplist_entries = $::redis::hash_max_ziplist_entries - $hash_max_ziplist_value = $::redis::hash_max_ziplist_value - $hz = $::redis::hz - $list_max_ziplist_entries = $::redis::list_max_ziplist_entries - $list_max_ziplist_value = $::redis::list_max_ziplist_value - $set_max_intset_entries = $::redis::set_max_intset_entries + $stop_writes_on_bgsave_error = $::redis::stop_writes_on_bgsave_error + $syslog_enabled = $::redis::syslog_enabled + $syslog_facility = $::redis::syslog_facility + $tcp_keepalive = $::redis::tcp_keepalive + $timeout = $::redis::timeout + $workdir = $::redis::workdir $zset_max_ziplist_entries = $::redis::zset_max_ziplist_entries $zset_max_ziplist_value = $::redis::zset_max_ziplist_value - $activerehashing = $::redis::activerehashing - $extra_config_file = $::redis::extra_config_file if $::redis::notify_service { File { diff --git a/redis/manifests/init.pp b/redis/manifests/init.pp index a420bddd3..66dd6f58e 100644 --- a/redis/manifests/init.pp +++ b/redis/manifests/init.pp @@ -23,7 +23,7 @@ # [*auto_aof_rewrite_min_size*] # Adjust minimum size for auto-aof-rewrite. # -# Default: 64min +# Default: 64mb # # [*auto_aof_rewrite_percentage*] # Adjust percentatge for auto-aof-rewrite. @@ -331,6 +331,12 @@ # # Default: 1024 # +# [*stop_writes_on_bgsave_error*] +# If false then Redis will continue to work as usual even if there +# are problems with disk, permissions, and so forth. +# +# Default: true +# # [*syslog_enabled*] # Enable/disable logging to the system logger. # @@ -342,6 +348,22 @@ # # Default: undef # +# [*tcp_keepalive*] +# TCP keepalive. +# +# If non-zero, use SO_KEEPALIVE to send TCP ACKs to clients in absence +# of communication. This is useful for two reasons: +# +# 1) Detect dead peers. +# 2) Take the connection alive from the point of view of network +# equipment in the middle. +# +# On Linux, the specified value (in seconds) is the period used to send ACKs. +# Note that to close the connection the double of the time is needed. +# On other kernels the period depends on the kernel configuration. +# +# A reasonable value for this option is 60 seconds. +# # [*timeout*] # Close the connection after a client is idle for N seconds (0 to disable). # @@ -368,6 +390,23 @@ # # Default: 64 # +# [*cluster_enabled*] +# Enables redis 3.0 cluster functionality +# +# Default: false +# +# [*cluster_config_file*] +# Config file for saving cluster nodes configuration. This file is never touched by humans. +# Only set if cluster_enabled is true +# +# Default: nodes.conf +# +# [*cluster_node_timeout*] +# Node timeout +# Only set if cluster_enabled is true +# +# Default: 5000 +# # == Actions: # - Install and configure Redis # @@ -439,13 +478,18 @@ $slaveof = $::redis::params::slaveof, $slowlog_log_slower_than = $::redis::params::slowlog_log_slower_than, $slowlog_max_len = $::redis::params::slowlog_max_len, + $stop_writes_on_bgsave_error = $::redis::params::stop_writes_on_bgsave_error, $syslog_enabled = $::redis::params::syslog_enabled, $syslog_facility = $::redis::params::syslog_facility, + $tcp_keepalive = $::redis::params::tcp_keepalive, $timeout = $::redis::params::timeout, $ulimit = $::redis::params::ulimit, $workdir = $::redis::params::workdir, $zset_max_ziplist_entries = $::redis::params::zset_max_ziplist_entries, $zset_max_ziplist_value = $::redis::params::zset_max_ziplist_value, + $cluster_enabled = $::redis::params::cluster_enabled, + $cluster_config_file = $::redis::params::cluster_config_file, + $cluster_node_timeout = $::redis::params::cluster_node_timeout, ) inherits redis::params { anchor { 'redis::begin': } anchor { 'redis::end': } diff --git a/redis/manifests/params.pp b/redis/manifests/params.pp index e4d7fc31e..d0697fc8e 100644 --- a/redis/manifests/params.pp +++ b/redis/manifests/params.pp @@ -10,7 +10,7 @@ $activerehashing = true $appendfsync = 'everysec' $appendonly = false - $auto_aof_rewrite_min_size = '64min' + $auto_aof_rewrite_min_size = '64mb' $auto_aof_rewrite_percentage = 100 $bind = '127.0.0.1' $conf_template = 'redis/redis.conf.erb' @@ -56,8 +56,10 @@ $set_max_intset_entries = 512 $slowlog_log_slower_than = 10000 $slowlog_max_len = 1024 + $stop_writes_on_bgsave_error = true $syslog_enabled = undef $syslog_facility = undef + $tcp_keepalive = 0 $timeout = 0 $ulimit = 65536 $workdir = '/var/lib/redis/' @@ -72,6 +74,11 @@ $slave_serve_stale_data = true $slaveof = undef + # redis.conf.erb - redis 3.0 clustering + $cluster_enabled = false + $cluster_config_file = 'nodes.conf' + $cluster_node_timeout = 5000 + case $::osfamily { 'Debian': { $config_dir = '/etc/redis' @@ -86,6 +93,7 @@ $package_name = 'redis-server' $sentinel_config_file = '/etc/redis/redis-sentinel.conf' $sentinel_config_file_orig = '/etc/redis/redis-sentinel.conf.puppet' + $sentinel_daemonize = true $sentinel_init_script = '/etc/init.d/redis-sentinel' $sentinel_package_name = 'redis-server' $sentinel_package_ensure = 'present' @@ -107,12 +115,13 @@ $config_file_mode = '0644' $config_group = 'root' $config_owner = 'redis' - $daemonize = false + $daemonize = true $log_dir_mode = '0755' $package_ensure = 'present' $package_name = 'redis' $sentinel_config_file = '/etc/redis-sentinel.conf' $sentinel_config_file_orig = '/etc/redis-sentinel.conf.puppet' + $sentinel_daemonize = false $sentinel_init_script = undef $sentinel_package_name = 'redis' $sentinel_package_ensure = 'present' @@ -140,6 +149,7 @@ $package_name = 'redis' $sentinel_config_file = '/usr/local/etc/redis-sentinel.conf' $sentinel_config_file_orig = '/usr/local/etc/redis-sentinel.conf.puppet' + $sentinel_daemonize = true $sentinel_init_script = undef $sentinel_package_name = 'redis' $sentinel_package_ensure = 'present' @@ -167,6 +177,7 @@ $package_name = 'redis' $sentinel_config_file = '/etc/redis/redis-sentinel.conf' $sentinel_config_file_orig = '/etc/redis/redis-sentinel.conf.puppet' + $sentinel_daemonize = true $sentinel_init_script = undef $sentinel_package_name = 'redis' $sentinel_package_ensure = 'present' diff --git a/redis/manifests/preinstall.pp b/redis/manifests/preinstall.pp index e7b5bd056..5f3ce6ae7 100644 --- a/redis/manifests/preinstall.pp +++ b/redis/manifests/preinstall.pp @@ -7,10 +7,10 @@ if $::redis::manage_repo { case $::operatingsystem { 'RedHat', 'CentOS', 'Scientific', 'OEL': { - if $::operatingsystemmajrelease < '7' { - $rpm_url = $::operatingsystemmajrelease ? { - '5' => "http://download.powerstack.org/5/${::architecture}/", - '6' => "http://download.powerstack.org/6/${::architecture}/", + if (versioncmp($::operatingsystemrelease, '7.0') == -1) { + $rpm_url = $::operatingsystemrelease ? { + /^5/ => "http://download.powerstack.org/5/${::architecture}/", + /^6/ => "http://download.powerstack.org/6/${::architecture}/", default => Fail['Operating system or release not supported.'], } @@ -29,7 +29,7 @@ } } - if $::operatingsystemmajrelease == '7' { + if (versioncmp($::operatingsystemmajrelease, '7') >= 0) { require ::epel } } diff --git a/redis/manifests/sentinel.pp b/redis/manifests/sentinel.pp index 552df9496..4f3663865 100644 --- a/redis/manifests/sentinel.pp +++ b/redis/manifests/sentinel.pp @@ -36,6 +36,11 @@ # # Default: redis/redis-sentinel.conf.erb # +# [*daemonize*] +# Have Redis sentinel run as a daemon. +# +# Default: true +# # [*down_after*] # Number of milliseconds the master (or any attached slave or sentinel) # should be unreachable (as in, not acceptable reply to PING, continuously, @@ -149,6 +154,7 @@ $config_file_orig = $::redis::params::sentinel_config_file_orig, $config_file_mode = $::redis::params::sentinel_config_file_mode, $conf_template = $::redis::params::sentinel_conf_template, + $daemonize = $::redis::params::sentinel_daemonize, $down_after = $::redis::params::sentinel_down_after, $failover_timeout = $::redis::params::sentinel_failover_timeout, $init_script = $::redis::params::sentinel_init_script, @@ -169,9 +175,8 @@ $working_dir = $::redis::params::sentinel_working_dir, $notification_script = $::redis::params::sentinel_notification_script, ) inherits redis::params { - $daemonize = $::redis::daemonize - unless defined(Package['$package_name']) { + unless defined(Package[$package_name]) { ensure_resource('package', $package_name, { 'ensure' => $package_ensure }) @@ -196,6 +201,7 @@ } if $init_script { + file { $init_script: ensure => present, @@ -205,10 +211,13 @@ content => template($init_template), require => Package[$package_name]; } + exec { '/usr/sbin/update-rc.d redis-sentinel defaults': - require => File[$init_script]; + subscribe => File[$init_script], + refreshonly => true; } + } service { $service_name: diff --git a/redis/metadata.json b/redis/metadata.json index 76da52ee6..f93c305e7 100644 --- a/redis/metadata.json +++ b/redis/metadata.json @@ -1,6 +1,6 @@ { "name": "arioch-redis", - "version": "1.1.3", + "version": "1.2.1", "author": "Tom De Vylder", "summary": "Redis module", "license": "Apache-2.0", diff --git a/redis/spec/acceptance/nodesets/default.yml b/redis/spec/acceptance/nodesets/default.yml new file mode 100644 index 000000000..780db9ef0 --- /dev/null +++ b/redis/spec/acceptance/nodesets/default.yml @@ -0,0 +1,11 @@ +HOSTS: + centos-65-x64: + roles: + - master + platform: el-6-x86_64 + box : centos-65-x64-vbox436-nocm + box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-nocm.box + hypervisor : vagrant + +CONFIG: + type: foss diff --git a/redis/spec/acceptance/redis_spec.rb b/redis/spec/acceptance/redis_spec.rb new file mode 100644 index 000000000..1e1998a7a --- /dev/null +++ b/redis/spec/acceptance/redis_spec.rb @@ -0,0 +1,23 @@ +require 'spec_helper_acceptance' + +describe 'redis' do + it 'should run successfully' do + pp = <<-EOS + Exec { + path => [ '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', ] + } + + class { '::redis': + manage_repo => true, + } + EOS + + # Apply twice to ensure no errors the second time. + apply_manifest(pp, :catch_failures => true) + apply_manifest(pp, :catch_changes => true) + end + + describe package('redis') do + it { should be_installed } + end +end diff --git a/redis/spec/classes/redis_spec.rb b/redis/spec/classes/redis_spec.rb index 97a159933..193d99936 100644 --- a/redis/spec/classes/redis_spec.rb +++ b/redis/spec/classes/redis_spec.rb @@ -699,6 +699,19 @@ } end + describe 'with parameter stop_writes_on_bgsave_error' do + let (:params) { + { + :stop_writes_on_bgsave_error => true + } + } + + it { should contain_file('/etc/redis/redis.conf').with( + 'content' => /stop-writes-on-bgsave-error.*yes/ + ) + } + end + describe 'with parameter syslog_enabled' do let (:params) { { @@ -726,6 +739,19 @@ } end + describe 'with parameter tcp_keepalive' do + let (:params) { + { + :tcp_keepalive => '_VALUE_' + } + } + + it { should contain_file('/etc/redis/redis.conf').with( + 'content' => /tcp-keepalive.*_VALUE_/ + ) + } + end + describe 'with parameter timeout' do let (:params) { { @@ -777,5 +803,60 @@ ) } end + + describe 'with parameter cluster_enabled-false' do + let (:params) { + { + :cluster_enabled => false + } + } + + it { should_not contain_file('/etc/redis/redis.conf').with( + 'content' => /cluster-enabled/ + ) + } + end + + describe 'with parameter cluster_enabled-true' do + let (:params) { + { + :cluster_enabled => true + } + } + + it { should contain_file('/etc/redis/redis.conf').with( + 'content' => /cluster-enabled.*yes/ + ) + } + end + + describe 'with parameter cluster_config_file' do + let (:params) { + { + :cluster_enabled => true, + :cluster_config_file => '_VALUE_' + } + } + + it { should contain_file('/etc/redis/redis.conf').with( + 'content' => /cluster-config-file.*_VALUE_/ + ) + } + end + + describe 'with parameter cluster_config_file' do + let (:params) { + { + :cluster_enabled => true, + :cluster_node_timeout => '_VALUE_' + } + } + + it { should contain_file('/etc/redis/redis.conf').with( + 'content' => /cluster-node-timeout.*_VALUE_/ + ) + } + end + end diff --git a/redis/spec/spec_helper_acceptance.rb b/redis/spec/spec_helper_acceptance.rb new file mode 100644 index 000000000..ac4f001d7 --- /dev/null +++ b/redis/spec/spec_helper_acceptance.rb @@ -0,0 +1,31 @@ +require 'beaker-rspec' + +unless ENV['RS_PROVISION'] == 'no' + hosts.each do |host| + # Install Puppet + if host.is_pe? + install_pe + else + install_puppet + end + end +end + +RSpec.configure do |c| + # Project root + proj_root = File.expand_path(File.join(File.dirname(__FILE__), '..')) + + # Readable test descriptions + c.formatter = :documentation + + c.before :suite do + # Install module and dependencies + puppet_module_install(:source => proj_root, :module_name => 'redis') + + hosts.each do |host| + shell("/bin/touch #{default['puppetpath']}/hiera.yaml") + + shell('puppet module install puppetlabs-stdlib', { :acceptable_exit_codes => [0,1] }) + end + end +end diff --git a/redis/templates/redis.conf.erb b/redis/templates/redis.conf.erb index 952207fce..e44542708 100644 --- a/redis/templates/redis.conf.erb +++ b/redis/templates/redis.conf.erb @@ -39,6 +39,22 @@ bind <%= @bind %> # Close the connection after a client is idle for N seconds (0 to disable) timeout <%= @timeout %> +# TCP keepalive. +# +# If non-zero, use SO_KEEPALIVE to send TCP ACKs to clients in absence +# of communication. This is useful for two reasons: +# +# 1) Detect dead peers. +# 2) Take the connection alive from the point of view of network +# equipment in the middle. +# +# On Linux, the specified value (in seconds) is the period used to send ACKs. +# Note that to close the connection the double of the time is needed. +# On other kernels the period depends on the kernel configuration. +# +# A reasonable value for this option is 60 seconds. +tcp-keepalive <%= @tcp_keepalive %> + # Set server verbosity to 'debug' # it can be one of: # debug (a lot of information, useful for development/testing) @@ -108,7 +124,7 @@ save 60 10000 # and persistence, you may want to disable this feature so that Redis will # continue to work as usually even if there are problems with disk, # permissions, and so forth. -stop-writes-on-bgsave-error yes +<% if @stop_writes_on_bgsave_error -%>stop-writes-on-bgsave-error yes<% else -%>stop-writes-on-bgsave-error no<% end -%> # Compress string objects using LZF when dump .rdb databases? # For default that's set to 'yes' as it's almost always a win. @@ -551,6 +567,14 @@ client-output-buffer-limit pubsub 32mb 8mb 60 # 100 only in environments where very low latency is required. hz <%= @hz %> +# Redis Cluster Settings +<% if @cluster_enabled -%> +cluster-enabled yes +cluster-config-file <%= @cluster_config_file %> +cluster-node-timeout <%= @cluster_node_timeout %> +<% end -%> + + ################################## INCLUDES ################################### # Include one or more other config files here. This is useful if you @@ -563,4 +587,3 @@ hz <%= @hz %> <% if @extra_config_file -%> include <%= @extra_config_file %> <% end -%> - diff --git a/sahara/CHANGELOG.md b/sahara/CHANGELOG.md index 9a769f084..79fe88816 100644 --- a/sahara/CHANGELOG.md +++ b/sahara/CHANGELOG.md @@ -1,3 +1,36 @@ +##2015-11-25 - 7.0.0 +###Summary + +This is a backwards-incompatible major release for OpenStack Liberty. + +####Backwards-incompatible changes +- remove kilo deprecated parameters + +####Features +- add distribute mode support +- introduce Sahara extended logging class +- remove POSIX file modes +- add tag to package and service resources +- add sahara::config class +- add sahara::db::sync +- add support of SSL +- add an ability to manage use_stderr parameter +- reflect provider change in puppet-openstacklib +- introduce sahara::db class +- db: use postgresql lib class for psycopg package +- add new parameter 'plugins' for Sahara +- configure rpc options separately from ceilometer notifications + +####Bugfixes +- rely on autorequire for config resource ordering +- remove Sqlite validation for database_connection + +####Maintenance +- initial msync run for all Puppet OpenStack modules +- try to use zuul-cloner to prepare fixtures +- remove class_parameter_defaults puppet-lint check +- acceptance: use common bits from puppet-openstack-integration + ##2015-10-10 - 6.1.0 ###Summary diff --git a/sahara/README.md b/sahara/README.md index 667762deb..a462c39f1 100644 --- a/sahara/README.md +++ b/sahara/README.md @@ -1,7 +1,7 @@ sahara ====== -6.1.0 - 2015.1 - Kilo +7.0.0 - 2015.2 - Liberty #### Table of Contents diff --git a/sahara/Rakefile b/sahara/Rakefile index ed79bead4..9692ffdd2 100644 --- a/sahara/Rakefile +++ b/sahara/Rakefile @@ -58,7 +58,7 @@ task :spec_prep do zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] sh(*zuul_clone_cmd) else - sh("git clone https://git.openstack.org/#{repo} #{repo}") + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") end script = ['env'] script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] diff --git a/sahara/manifests/init.pp b/sahara/manifests/init.pp index 176bdd381..d226704c2 100644 --- a/sahara/manifests/init.pp +++ b/sahara/manifests/init.pp @@ -472,6 +472,8 @@ if $rpc_backend == 'qpid' { + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') + if $qpid_hosts { sahara_config { 'oslo_messaging_qpid/qpid_hosts': value => join($qpid_hosts, ','); diff --git a/sahara/manifests/notify/qpid.pp b/sahara/manifests/notify/qpid.pp index e1c95d8d1..8c6d812d4 100644 --- a/sahara/manifests/notify/qpid.pp +++ b/sahara/manifests/notify/qpid.pp @@ -105,6 +105,7 @@ warning('This class is deprecated. Use sahara::init for configuration rpc options instead') warning('This class is deprecated. Use sahara::notify for configuration ceilometer notifications instead') + warning('Qpid driver is removed from Oslo.messaging in the Mitaka release') if $kombu_ssl_version { warning('The kombu_ssl_version parameter is deprecated and has no effect.') diff --git a/sahara/metadata.json b/sahara/metadata.json index ee52a0d43..bb878f28a 100644 --- a/sahara/metadata.json +++ b/sahara/metadata.json @@ -1,6 +1,6 @@ { "name": "openstack-sahara", - "version": "6.1.0", + "version": "7.0.0", "author": "Red Hat and OpenStack Contributors", "summary": "Puppet module for OpenStack Sahara", "license": "Apache-2.0", @@ -28,10 +28,10 @@ "description": "Installs and configures OpenStack Sahara (Data Processing).", "dependencies": [ { "name": "duritong/sysctl", "version_requirement": ">=0.0.1 <1.0.0" }, - { "name": "openstack/keystone", "version_requirement": ">=6.0.0 <7.0.0" }, + { "name": "openstack/keystone", "version_requirement": ">=7.0.0 <8.0.0" }, { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, { "name": "puppetlabs/postgresql", "version_requirement": ">=3.0.0" }, - { "name": "openstack/openstacklib", "version_requirement": ">=6.0.0 <7.0.0" } + { "name": "openstack/openstacklib", "version_requirement": ">=7.0.0 <8.0.0" } ] } diff --git a/sahara/spec/spec_helper_acceptance.rb b/sahara/spec/spec_helper_acceptance.rb index 429e807c4..15e8cc977 100644 --- a/sahara/spec/spec_helper_acceptance.rb +++ b/sahara/spec/spec_helper_acceptance.rb @@ -38,7 +38,7 @@ zuul_clone_cmd += "git://git.openstack.org #{repo}" on host, zuul_clone_cmd else - on host, "git clone https://git.openstack.org/#{repo} #{repo}" + on host, "git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}" end on host, "ZUUL_REF=#{zuul_ref} ZUUL_BRANCH=#{zuul_branch} ZUUL_URL=#{zuul_url} bash #{repo}/install_modules.sh" diff --git a/sensu/README.md b/sensu/README.md index e6837108d..24720373d 100644 --- a/sensu/README.md +++ b/sensu/README.md @@ -69,7 +69,7 @@ To quickly try out Sensu, spin up a test virtual machine with Vagrant that alrea You can then access the API. - $ curl http://admin:secret@localhost:4567/info + $ curl http://admin:secret@192.168.56.10:4567/info Navigate to `192.168.56.10:3000` to use the uchiwa dashboard diff --git a/sensu/lib/puppet/provider/sensu_client_config/json.rb b/sensu/lib/puppet/provider/sensu_client_config/json.rb index 467be4017..8456bd96b 100644 --- a/sensu/lib/puppet/provider/sensu_client_config/json.rb +++ b/sensu/lib/puppet/provider/sensu_client_config/json.rb @@ -90,7 +90,7 @@ def keepalive end def keepalive=(value) - conf['client']['keepalive'] = value + conf['client']['keepalive'] = to_type(value) end def safe_mode diff --git a/sensu/lib/puppet/provider/sensu_enterprise_dashboard_config/json.rb b/sensu/lib/puppet/provider/sensu_enterprise_dashboard_config/json.rb index 78e41d3cc..8f0a5edf5 100644 --- a/sensu/lib/puppet/provider/sensu_enterprise_dashboard_config/json.rb +++ b/sensu/lib/puppet/provider/sensu_enterprise_dashboard_config/json.rb @@ -109,15 +109,15 @@ def user=(value) # Public: Retrieve the password for the Dashboard # # Returns the String password. - def password - conf['dashboard']['password'] + def pass + conf['dashboard']['pass'] end # Public: Set the Dashboard password # # Returns nothing. - def password=(value) - conf['dashboard']['password'] = value + def pass=(value) + conf['dashboard']['pass'] = value end # Public: Retrieve the Github config diff --git a/sensu/lib/puppet/type/sensu_client_config.rb b/sensu/lib/puppet/type/sensu_client_config.rb index 1c8300289..aaf6c697d 100644 --- a/sensu/lib/puppet/type/sensu_client_config.rb +++ b/sensu/lib/puppet/type/sensu_client_config.rb @@ -49,12 +49,8 @@ def insync?(is) desc "A set of attributes that configure the Sensu client socket." include PuppetX::Sensu::ToType - def is_to_s(hash = @is) - hash.keys.sort.map {|key| "#{key} => #{hash[key]}"}.join(", ") - end - - def should_to_s(hash = @should) - hash.keys.sort.map {|key| "#{key} => #{hash[key]}"}.join(", ") + munge do |value| + value.each { |k, v| value[k] = to_type(v) } end def insync?(is) @@ -88,6 +84,10 @@ def insync?(is) include PuppetX::Sensu::ToType + munge do |value| + value.each { |k, v| value[k] = to_type(v) } + end + def is_to_s(hash = @is) hash.keys.sort.map {|key| "#{key} => #{hash[key]}"}.join(", ") end @@ -116,6 +116,10 @@ def insync?(is) include PuppetX::Sensu::ToType + munge do |value| + value.each { |k, v| value[k] = to_type(v) } + end + def is_to_s(hash = @is) hash.keys.sort.map {|key| "#{key} => #{hash[key]}"}.join(", ") end diff --git a/sensu/manifests/repo/apt.pp b/sensu/manifests/repo/apt.pp index 80c95163d..d565ef7eb 100644 --- a/sensu/manifests/repo/apt.pp +++ b/sensu/manifests/repo/apt.pp @@ -36,6 +36,13 @@ 'source' => $sensu::repo_key_source, }, before => Package['sensu'], + notify => Exec['apt-update'], + } + + exec { + 'apt-update': + refreshonly => true, + command => '/usr/bin/apt-get update'; } if $sensu::enterprise { diff --git a/sensu/tests/provision_client.sh b/sensu/tests/provision_client.sh index 9ed969199..74011db02 100644 --- a/sensu/tests/provision_client.sh +++ b/sensu/tests/provision_client.sh @@ -5,7 +5,7 @@ # apt-get install -y python-software-properties wget --quiet http://apt.puppetlabs.com/puppetlabs-release-precise.deb -O /tmp/puppetlabs-release-precise.deb dpkg -i /tmp/puppetlabs-release-precise.deb -apt-get update -apt-get install -y ruby-json puppet-common #masterless puppet +apt-get update +apt-get install -y ruby-json puppet-common ruby-dev #masterless puppet sed -i '/templatedir/d' /etc/puppet/puppet.conf puppet module install sensu/sensu diff --git a/sensu/tests/provision_server.sh b/sensu/tests/provision_server.sh index e8e92fb1f..d9f15a305 100644 --- a/sensu/tests/provision_server.sh +++ b/sensu/tests/provision_server.sh @@ -5,8 +5,8 @@ # apt-get install -y python-software-properties wget --quiet http://apt.puppetlabs.com/puppetlabs-release-precise.deb -O /tmp/puppetlabs-release-precise.deb dpkg -i /tmp/puppetlabs-release-precise.deb -apt-get update -apt-get install -y ruby-json redis-server puppet-common #masterless puppet +apt-get update +apt-get install -y ruby-json redis-server puppet-common ruby-dev #masterless puppet sed -i '/templatedir/d' /etc/puppet/puppet.conf puppet module install sensu/sensu puppet module install puppetlabs/rabbitmq diff --git a/tripleo/.fixtures.yml b/tripleo/.fixtures.yml index e2444d17f..e3ab8f9cd 100644 --- a/tripleo/.fixtures.yml +++ b/tripleo/.fixtures.yml @@ -2,26 +2,5 @@ fixtures: repositories: 'firewall': 'git://github.com/puppetlabs/puppetlabs-firewall.git' 'stdlib': 'git://github.com/puppetlabs/puppetlabs-stdlib.git' - 'midonet': - repo: 'git://github.com/midonet/puppet-midonet.git' - ref: 'v2015.06.7' - 'tomcat': - repo: 'git://github.com/puppetlabs/puppetlabs-tomcat.git' - ref: '1.3.2' - 'inifile': - repo: 'git://github.com/puppetlabs/puppetlabs-inifile.git' - ref: '1.4.2' - 'cassandra': - repo: 'git://github.com/midonet/puppet-cassandra.git' - ref: 'v1.1.1' - 'zookeeper': - repo: 'git://github.com/deric/puppet-zookeeper.git' - ref: 'v0.3.9' - 'datacat': - repo: 'git://github.com/richardc/puppet-datacat' - ref: '0.6.2' - 'java': - repo: 'git://github.com/puppetlabs/puppetlabs-java' - ref: '1.4.2' symlinks: "tripleo": "#{source_dir}" diff --git a/tripleo/.gitreview b/tripleo/.gitreview index 90b3af94c..14bd51b6e 100644 --- a/tripleo/.gitreview +++ b/tripleo/.gitreview @@ -2,3 +2,4 @@ host=review.openstack.org port=29418 project=openstack/puppet-tripleo.git +defaultbranch=stable/liberty diff --git a/tripleo/lib/puppet/parser/functions/extract_id.rb b/tripleo/lib/puppet/parser/functions/extract_id.rb deleted file mode 100644 index 61734abfa..000000000 --- a/tripleo/lib/puppet/parser/functions/extract_id.rb +++ /dev/null @@ -1,14 +0,0 @@ -# Custom function to extract the index from a list. -# The list are a list of hostname, and the index is the n'th -# position of the host in list -module Puppet::Parser::Functions - newfunction(:extract_id, :type => :rvalue) do |argv| - hosts = argv[0] - if hosts.class != Array - hosts = [hosts] - end - hostname = argv[1] - hash = Hash[hosts.map.with_index.to_a] - return hash[hostname].to_i + 1 - end -end diff --git a/tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb b/tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb deleted file mode 100644 index 814326e84..000000000 --- a/tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb +++ /dev/null @@ -1,24 +0,0 @@ -# Custom function to convert a list of ips to a map -# like {'ip' => xxx.xxx.xxx.xxx }. This function is needed -# because a not-so-good design of the puppet-midonet module -# and we hope to deprecate it soon. - -module Puppet::Parser::Functions - newfunction(:list_to_zookeeper_hash, :type => :rvalue, :doc => <<-EOS - This function returns Zookeper configuration list of hash - EOS - ) do |argv| - zk_list = argv[0] - if zk_list.class != Array - zk_list = [zk_list] - end - result = Array.new - zk_list.each do |zk_ip| - zk_map = Hash.new - zk_map['ip'] = zk_ip - zk_map['port'] = 2181 - result.push(zk_map) - end - return result - end -end diff --git a/tripleo/manifests/cluster/cassandra.pp b/tripleo/manifests/cluster/cassandra.pp deleted file mode 100644 index a810ccd79..000000000 --- a/tripleo/manifests/cluster/cassandra.pp +++ /dev/null @@ -1,69 +0,0 @@ -# -# Copyright (C) 2015 Midokura SARL -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may -# not use this file except in compliance with the License. You may obtain -# a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. -# -# == Class: tripleo::cluster::cassandra -# -# Deploys a cassandra service that belongs to a cluster. Uses puppet-cassandra -# -# == Parameters: -# -# [*cassandra_servers*] -# (required) All the IP addresses of the cassandra cluster. -# Array of strings value. -# -# [*cassandra_ip*] -# (required) IP address of the current host. -# String value -# -# [*storage_port*] -# (optional) Inter-node cluster communication port. -# Defaults to 7000. -# -# [*ssl_storage_port*] -# (optional) SSL Inter-node cluster communication port. -# Defaults to 7001. -# -# [*client_port*] -# (optional) Cassandra client port. -# Defaults to 9042. -# -# [*client_port_thrift*] -# (optional) Cassandra client port thrift. -# Defaults to 9160. -# -class tripleo::cluster::cassandra( - $cassandra_servers, - $cassandra_ip, - $storage_port = '7000', - $ssl_storage_port = '7001', - $client_port = '9042', - $client_port_thrift = '9160' -) -{ - validate_array($cassandra_servers) - validate_ipv4_address($cassandra_ip) - - class {'::cassandra::run': - seeds => $cassandra_servers, - seed_address => $cassandra_ip, - conf_dir => '/etc/cassandra/default.conf', - pid_dir => '/var/run/cassandra', - service_path => '/sbin', - storage_port => $storage_port, - ssl_storage_port => $ssl_storage_port, - client_port => $client_port, - client_port_thrift => $client_port_thrift - } -} diff --git a/tripleo/manifests/cluster/zookeeper.pp b/tripleo/manifests/cluster/zookeeper.pp deleted file mode 100644 index 6f4adbcd8..000000000 --- a/tripleo/manifests/cluster/zookeeper.pp +++ /dev/null @@ -1,65 +0,0 @@ -# -# Copyright (C) 2015 Midokura SARL -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may -# not use this file except in compliance with the License. You may obtain -# a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. -# -# == Class: tripleo::cluster::zookeeper -# -# Deploys a zookeeper service that belongs to a cluster. Uses deric-zookeeper -# -# == Parameters: -# -# [*zookeeper_server_ips*] -# (required) List of IP addresses of the zookeeper cluster. -# Arrays of strings value. -# -# [*zookeeper_client_ip*] -# (required) IP address of the host where zookeeper will listen IP addresses. -# String (IPv4) value. -# -# [*zookeeper_hostnames*] -# (required) List of hostnames of the zookeeper cluster. The hostname of the -# node will be used to define the ID of the zookeeper configuration -# Array of strings value. -# - -class tripleo::cluster::zookeeper( - $zookeeper_server_ips, - $zookeeper_client_ip, - $zookeeper_hostnames -) -{ - validate_array($zookeeper_server_ips) - validate_ipv4_address($zookeeper_client_ip) - validate_array($zookeeper_hostnames) - - # TODO(devvesa) Zookeeper package should provide these paths, - # remove this lines as soon as it will. - file {['/usr/lib', '/usr/lib/zookeeper', '/usr/lib/zookeeper/bin/']: - ensure => directory - } - - file {'/usr/lib/zookeeper/bin/zkEnv.sh': - ensure => link, - target => '/usr/libexec/zkEnv.sh' - } - - class {'::zookeeper': - servers => $zookeeper_server_ips, - client_ip => $zookeeper_client_ip, - id => extract_id($zookeeper_hostnames, $::hostname), - cfg_dir => '/etc/zookeeper/conf', - } - - File['/usr/lib/zookeeper/bin/zkEnv.sh'] -> Class['::zookeeper'] -} diff --git a/tripleo/manifests/loadbalancer.pp b/tripleo/manifests/loadbalancer.pp index e34b01e4b..16a4f80a9 100644 --- a/tripleo/manifests/loadbalancer.pp +++ b/tripleo/manifests/loadbalancer.pp @@ -814,7 +814,6 @@ bind => $nova_novnc_bind_opts, options => { 'balance' => 'source', - 'timeout' => [ 'tunnel 1h' ], }, collect_exported => false, } diff --git a/tripleo/manifests/network/midonet/agent.pp b/tripleo/manifests/network/midonet/agent.pp deleted file mode 100644 index 2f1da9414..000000000 --- a/tripleo/manifests/network/midonet/agent.pp +++ /dev/null @@ -1,61 +0,0 @@ -# -# Copyright (C) 2015 Midokura SARL -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may -# not use this file except in compliance with the License. You may obtain -# a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. -# -# == Class: tripleo::network::midonet::agent -# -# Configure the midonet agent -# -# == Parameters: -# -# [*zookeeper_servers*] -# (required) List of IPs of the zookeeper server cluster. It will configure -# the connection using the 2181 port. -# Array of strings value. -# -# [*cassandra_seeds*] -# (required) List of IPs of the cassandra cluster. -# Array of strings value. -# -class tripleo::network::midonet::agent ( - $zookeeper_servers, - $cassandra_seeds -) { - - validate_array($zookeeper_servers) - validate_array($cassandra_seeds) - - # FIXME: This statement should be controlled by hiera on heat templates - # project - # Make sure openvswitch service is not running - service {'openvswitch': - ensure => stopped, - enable => false - } - - exec {'delete datapaths': - command => '/usr/bin/mm-dpctl --delete-dp ovs-system', - path => '/usr/bin:/usr/sbin:/bin', - onlyif => '/usr/bin/mm-dpctl --show-dp ovs-system' - } - - # Configure and run the agent - class {'::midonet::midonet_agent': - zk_servers => list_to_zookeeper_hash($zookeeper_servers), - cassandra_seeds => $cassandra_seeds - } - - Service['openvswitch'] -> Class['::midonet::midonet_agent::run'] - Exec['delete datapaths'] -> Class['::midonet::midonet_agent::run'] -} diff --git a/tripleo/manifests/network/midonet/api.pp b/tripleo/manifests/network/midonet/api.pp deleted file mode 100644 index 33b22178a..000000000 --- a/tripleo/manifests/network/midonet/api.pp +++ /dev/null @@ -1,117 +0,0 @@ -# -# Copyright (C) 2015 Midokura SARL -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may -# not use this file except in compliance with the License. You may obtain -# a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. -# -# == Class: tripleo::network::midonet::api -# -# Configure the MidoNet API -# -# == Parameters: -# -# [*zookeeper_servers*] -# (required) List IPs of the zookeeper server cluster. Zookeeper is the -# backend database where MidoNet stores the virtual network topology. -# Array of strings value. -# -# [*vip*] -# (required) Public Virtual IP where the API will be exposed. -# String (IPv4) value. -# -# [*keystone_ip*] -# (required) MidoNet API is registered as an OpenStack service. Provide the -# keystone ip address. -# String (IPv4) value. -# -# [*keystone_admin_token*] -# (required) MidoNet API is registered as an OpenStack service. It needs the -# keystone admin token to perform some admin calls. -# String value. -# -# [*bind_address*] -# (required) MidoNet API uses a Tomcat instance to offer the REST service. The -# ip address where to bind the tomcat service. -# String (IPv4) value. -# -# [*admin_password*] -# (required) OpenStack admin user password. -# String value. -# -# [*keystone_port*] -# (optional) MidoNet API is registered as an OpenStack service. Provide -# the keystone port. -# Defaults to 35357 -# -# [*keystone_tenant_name*] -# (optional) Tenant of the keystone service. -# Defaults to 'admin' -# -# [*admin_user_name*] -# (optional) OpenStack admin user name. -# Defaults to 'admin' -# -# [*admin_tenant_name*] -# (optional). OpenStack admin tenant name. -# Defaults to 'admin' -# - -class tripleo::network::midonet::api( - $zookeeper_servers, - $vip, - $keystone_ip, - $keystone_admin_token, - $bind_address, - $admin_password, - $keystone_port = 35357, - $keystone_tenant_name = 'admin', - $admin_user_name = 'admin', - $admin_tenant_name = 'admin' -) -{ - validate_array($zookeeper_servers) - validate_ip_address($vip) - validate_ip_address($keystone_ip) - validate_ip_address($bind_address) - - # Run Tomcat and MidoNet API - class {'::tomcat': - install_from_source => false - } -> - - package {'midonet-api': - ensure => present - } -> - - class {'::midonet::midonet_api::run': - zk_servers => list_to_zookeeper_hash($zookeeper_servers), - keystone_auth => true, - tomcat_package => 'tomcat', - vtep => false, - api_ip => $vip, - api_port => '8081', - keystone_host => $keystone_ip, - keystone_port => $keystone_port, - keystone_admin_token => $keystone_admin_token, - keystone_tenant_name => $keystone_tenant_name, - catalina_base => '/usr/share/tomcat', - bind_address => $bind_address - } - - # Configure the CLI - class {'::midonet::midonet_cli': - api_endpoint => "http://${vip}:8081/midonet-api", - username => $admin_user_name, - password => $admin_password, - tenant_name => $admin_tenant_name - } -} diff --git a/tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb b/tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb deleted file mode 100644 index 4f2eb6c0f..000000000 --- a/tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb +++ /dev/null @@ -1,45 +0,0 @@ -# -# Copyright (C) 2015 Midokura SARL -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may -# not use this file except in compliance with the License. You may obtain -# a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. -# -# Unit tests for the cassandra service - -require 'spec_helper' - -describe 'tripleo::cluster::cassandra' do - - shared_examples_for 'cassandra cluster service' do - - let :params do - { - :cassandra_servers => ['192.168.2.2', '192.168.2.3'], - :cassandra_ip => '192.168.2.2' - } - end - - it 'should configure cassandra' do - is_expected.to contain_class('cassandra::run').with( - :seeds => ['192.168.2.2', '192.168.2.3'], - :seed_address => '192.168.2.2', - :storage_port => '7000', - :ssl_storage_port => '7001', - :client_port => '9042', - :client_port_thrift => '9160' - ) - end - end - - it_configures 'cassandra cluster service' - -end diff --git a/tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb b/tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb deleted file mode 100644 index ed46164a4..000000000 --- a/tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb +++ /dev/null @@ -1,115 +0,0 @@ -# -# Copyright (C) 2015 Midokura SARL -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may -# not use this file except in compliance with the License. You may obtain -# a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. -# -# Unit tests for the zookeeper service - -require 'spec_helper' - -describe 'tripleo::cluster::zookeeper' do - - let :default_params do - { - :zookeeper_server_ips => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], - :zookeeper_hostnames => ['host1.midonet', 'host2.midonet', 'host3.midonet'] - } - end - - context 'on host1' do - let :facts do - { - :hostname => 'host1.midonet', - :osfamily => 'RedHat', - :operatingsystemmajrelease => 7, - } - end - - let :params do - { - :zookeeper_client_ip => '23.43.2.34' - } - end - - before do - params.merge!(default_params) - end - - it 'should call zookeeper using id==1' do - is_expected.to contain_class('zookeeper').with( - :servers => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], - :client_ip => '23.43.2.34', - :id => 1 - ) - end - - end - - context 'on host2' do - let :facts do - { - :hostname => 'host2.midonet', - :osfamily => 'RedHat', - :operatingsystemmajrelease => 7, - } - end - - let :params do - { - :zookeeper_client_ip => '23.43.2.35' - } - end - - before do - params.merge!(default_params) - end - - it 'should call zookeeper using id==1' do - is_expected.to contain_class('zookeeper').with( - :servers => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], - :client_ip => '23.43.2.35', - :id => 2 - ) - end - end - - context 'on host3' do - let :facts do - { - :hostname => 'host3.midonet', - :osfamily => 'RedHat', - :operatingsystemmajrelease => 7, - } - end - - let :params do - { - :zookeeper_client_ip => '23.43.2.36' - } - end - - before do - params.merge!(default_params) - end - - it 'should call zookeeper using id==1' do - is_expected.to contain_class('zookeeper').with( - :servers => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], - :client_ip => '23.43.2.36', - :id => 3 - ) - end - - end - -end diff --git a/tripleo/spec/classes/tripleo_firewall_spec.rb b/tripleo/spec/classes/tripleo_firewall_spec.rb index 7d1d1ecb8..c1249b9fa 100644 --- a/tripleo/spec/classes/tripleo_firewall_spec.rb +++ b/tripleo/spec/classes/tripleo_firewall_spec.rb @@ -105,10 +105,7 @@ context 'on RedHat platforms' do let :facts do - { - :osfamily => 'RedHat', - :operatingsystemrelease => '7.1', - } + { :osfamily => 'RedHat' } end it_configures 'tripleo node' diff --git a/tripleo/spec/classes/tripleo_midonet_agent_spec.rb b/tripleo/spec/classes/tripleo_midonet_agent_spec.rb deleted file mode 100644 index eb3abfe85..000000000 --- a/tripleo/spec/classes/tripleo_midonet_agent_spec.rb +++ /dev/null @@ -1,62 +0,0 @@ -# -# Copyright (C) 2015 Midokura SARL -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may -# not use this file except in compliance with the License. You may obtain -# a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. -# -# Unit tests for the midonet agent - -require 'spec_helper' - -describe 'tripleo::network::midonet::agent' do - - let :facts do - { - :hostname => 'host2.midonet', - :osfamily => 'RedHat', - :operatingsystem => 'CentOS', - :operatingsystemrelease => '7.1', - :operatingsystemmajrelease => 7, - } - end - - shared_examples_for 'midonet agent test' do - - let :params do - { - :zookeeper_servers => ['192.168.2.2', '192.168.2.3'], - :cassandra_seeds => ['192.168.2.2', '192.168.2.3'] - } - end - - it 'should stop openvswitch' do - is_expected.to contain_service('openvswitch').with( - :ensure => 'stopped', - :enable => false - ) - end - - it 'should run the agent with a list of maps' do - is_expected.to contain_class('midonet::midonet_agent').with( - :zk_servers => [{'ip' => '192.168.2.2', - 'port' => 2181}, - {'ip' => '192.168.2.3', - 'port' => 2181}], - :cassandra_seeds => ['192.168.2.2','192.168.2.3'] - ) - end - end - - it_configures 'midonet agent test' - - -end diff --git a/tripleo/spec/classes/tripleo_midonet_api_spec.rb b/tripleo/spec/classes/tripleo_midonet_api_spec.rb deleted file mode 100644 index 4b4729494..000000000 --- a/tripleo/spec/classes/tripleo_midonet_api_spec.rb +++ /dev/null @@ -1,72 +0,0 @@ -# -# Copyright (C) 2015 Midokura SARL -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may -# not use this file except in compliance with the License. You may obtain -# a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. -# -# Unit tests for the midonet api - -require 'spec_helper' - -describe 'tripleo::network::midonet::api' do - - let :facts do - { - :augeasversion => '1.0.0' - } - end - - shared_examples_for 'midonet api test' do - - let :params do - { - :zookeeper_servers => ['192.168.2.1', '192.168.2.2'], - :vip => '192.23.0.2', - :keystone_ip => '192.23.0.2', - :keystone_admin_token => 'admin_token', - :admin_password => 'admin_password', - :bind_address => '192.23.0.65' - } - end - - it 'should call api configuration' do - is_expected.to contain_class('midonet::midonet_api::run').with( - :zk_servers => [{'ip' => '192.168.2.1', 'port' => 2181}, - {'ip' => '192.168.2.2', 'port' => 2181}], - :keystone_auth => true, - :tomcat_package => 'tomcat', - :vtep => false, - :api_ip => '192.23.0.2', - :api_port => '8081', - :keystone_host => '192.23.0.2', - :keystone_port => 35357, - :keystone_admin_token => 'admin_token', - :keystone_tenant_name => 'admin', - :catalina_base => '/usr/share/tomcat', - :bind_address => '192.23.0.65' - ) - end - - it 'should install the cli' do - is_expected.to contain_class('midonet::midonet_cli').with( - :api_endpoint => 'http://192.23.0.2:8081/midonet-api', - :username => 'admin', - :password => 'admin_password', - :tenant_name => 'admin' - ) - end - - end - - it_configures 'midonet api test' - -end diff --git a/tripleo/spec/spec_helper.rb b/tripleo/spec/spec_helper.rb index 5cf9642eb..700be6a27 100644 --- a/tripleo/spec/spec_helper.rb +++ b/tripleo/spec/spec_helper.rb @@ -2,13 +2,9 @@ require 'shared_examples' require 'webmock/rspec' -fixture_path = File.expand_path(File.join(__FILE__, '..', 'fixtures')) - RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' c.alias_it_should_behave_like_to :it_raises, 'raises' - c.module_path = File.join(fixture_path, 'modules') - c.manifest_dir = File.join(fixture_path, 'manifests') c.default_facts = { :kernel => 'Linux', diff --git a/uchiwa/.gitignore b/uchiwa/.gitignore index 5104b2209..ef7293f7c 100755 --- a/uchiwa/.gitignore +++ b/uchiwa/.gitignore @@ -4,3 +4,4 @@ pkg spec/fixtures .rspec_system .vagrant +.bundle diff --git a/uchiwa/CHANGELOG.md b/uchiwa/CHANGELOG.md new file mode 100644 index 000000000..13ae84a72 --- /dev/null +++ b/uchiwa/CHANGELOG.md @@ -0,0 +1,117 @@ +# Change Log + +## [v1.0.1](https://github.com/Yelp/puppet-uchiwa/tree/v1.0.1) (2015-12-07) +[Full Changelog](https://github.com/Yelp/puppet-uchiwa/compare/v1.0.0...v1.0.1) + +**Closed issues:** + +- Changelog [\#30](https://github.com/Yelp/puppet-uchiwa/issues/30) + +**Merged pull requests:** + +- Update apt key id [\#62](https://github.com/Yelp/puppet-uchiwa/pull/62) ([queeno](https://github.com/queeno)) + +## [v1.0.0](https://github.com/Yelp/puppet-uchiwa/tree/v1.0.0) (2015-12-04) +[Full Changelog](https://github.com/Yelp/puppet-uchiwa/compare/0.3.0...v1.0.0) + +**Closed issues:** + +- Support the users attribute [\#46](https://github.com/Yelp/puppet-uchiwa/issues/46) +- Duplicate Apt Resource [\#36](https://github.com/Yelp/puppet-uchiwa/issues/36) +- JSON acceptance tests are failing [\#33](https://github.com/Yelp/puppet-uchiwa/issues/33) +- New puppet-uchiwa release? [\#25](https://github.com/Yelp/puppet-uchiwa/issues/25) + +**Merged pull requests:** + +- Introducing users param in puppet-uchiwa [\#61](https://github.com/Yelp/puppet-uchiwa/pull/61) ([queeno](https://github.com/queeno)) +- Update sensu apt repo. [\#59](https://github.com/Yelp/puppet-uchiwa/pull/59) ([liamjbennett](https://github.com/liamjbennett)) +- Update apt::source to use puppetlabs-apt 2.0.0+ [\#56](https://github.com/Yelp/puppet-uchiwa/pull/56) ([zxjinn](https://github.com/zxjinn)) +- add a test for sensu\_api\_endpoints param [\#54](https://github.com/Yelp/puppet-uchiwa/pull/54) ([somic](https://github.com/somic)) +- Changed example code from manage\_repo to install\_repo [\#52](https://github.com/Yelp/puppet-uchiwa/pull/52) ([standaloneSA](https://github.com/standaloneSA)) +- Try to cleanup doc and un-nest api definitions [\#51](https://github.com/Yelp/puppet-uchiwa/pull/51) ([solarkennedy](https://github.com/solarkennedy)) +- Properly type port number [\#50](https://github.com/Yelp/puppet-uchiwa/pull/50) ([DoriftoShoes](https://github.com/DoriftoShoes)) +- fix lint [\#44](https://github.com/Yelp/puppet-uchiwa/pull/44) ([bleuchtang](https://github.com/bleuchtang)) +- Fix warning when installing sensu apt repository [\#41](https://github.com/Yelp/puppet-uchiwa/pull/41) ([KoeSystems](https://github.com/KoeSystems)) +- apt module 2.0.0 breaks apt::source extensively [\#39](https://github.com/Yelp/puppet-uchiwa/pull/39) ([bobtfish](https://github.com/bobtfish)) +- Fixing Erroneous validations [\#37](https://github.com/Yelp/puppet-uchiwa/pull/37) ([magmax](https://github.com/magmax)) +- Fix tests [\#35](https://github.com/Yelp/puppet-uchiwa/pull/35) ([pauloconnor](https://github.com/pauloconnor)) +- Notify uchiwa service when package changes [\#34](https://github.com/Yelp/puppet-uchiwa/pull/34) ([timmow](https://github.com/timmow)) +- Ensure uchiwa.json is not world-readable [\#32](https://github.com/Yelp/puppet-uchiwa/pull/32) ([nhinds](https://github.com/nhinds)) +- Fix some issues with 3.7.\* Puppet [\#31](https://github.com/Yelp/puppet-uchiwa/pull/31) ([tayzlor](https://github.com/tayzlor)) +- missing default host for sensu API [\#29](https://github.com/Yelp/puppet-uchiwa/pull/29) ([n1tr0g](https://github.com/n1tr0g)) + +## [0.3.0](https://github.com/Yelp/puppet-uchiwa/tree/0.3.0) (2014-12-11) +[Full Changelog](https://github.com/Yelp/puppet-uchiwa/compare/0.2.7...0.3.0) + +**Closed issues:** + +- Obscure Datacat errors from Uchiwa module [\#24](https://github.com/Yelp/puppet-uchiwa/issues/24) + +**Merged pull requests:** + +- Remove artifact of PR \#21 [\#28](https://github.com/Yelp/puppet-uchiwa/pull/28) ([jbussdieker](https://github.com/jbussdieker)) +- Add the insecure parameter to the default endpoint [\#27](https://github.com/Yelp/puppet-uchiwa/pull/27) ([jbussdieker](https://github.com/jbussdieker)) +- Fix JSON syntax [\#26](https://github.com/Yelp/puppet-uchiwa/pull/26) ([jbussdieker](https://github.com/jbussdieker)) +- The refresh param now takes seconds rather than ms [\#23](https://github.com/Yelp/puppet-uchiwa/pull/23) ([queeno](https://github.com/queeno)) +- Remove the stats param from the puppet module [\#22](https://github.com/Yelp/puppet-uchiwa/pull/22) ([queeno](https://github.com/queeno)) +- Remove uchiwa::api and pass API data via class params [\#21](https://github.com/Yelp/puppet-uchiwa/pull/21) ([queeno](https://github.com/queeno)) +- Fix puppet-uchiwa beaker tests [\#20](https://github.com/Yelp/puppet-uchiwa/pull/20) ([queeno](https://github.com/queeno)) + +## [0.2.7](https://github.com/Yelp/puppet-uchiwa/tree/0.2.7) (2014-11-10) +[Full Changelog](https://github.com/Yelp/puppet-uchiwa/compare/0.2.6...0.2.7) + +**Closed issues:** + +- Am I being inappropriately redirected? [\#18](https://github.com/Yelp/puppet-uchiwa/issues/18) +- uchiwa is not under the yelp namespace on the forge. [\#16](https://github.com/Yelp/puppet-uchiwa/issues/16) + +**Merged pull requests:** + +- Add beaker tests [\#17](https://github.com/Yelp/puppet-uchiwa/pull/17) ([petems](https://github.com/petems)) + +## [0.2.6](https://github.com/Yelp/puppet-uchiwa/tree/0.2.6) (2014-09-25) +[Full Changelog](https://github.com/Yelp/puppet-uchiwa/compare/0.2.5...0.2.6) + +**Closed issues:** + +- Explanation of "incompatible with previous versions of the Sensu-Puppet module" [\#14](https://github.com/Yelp/puppet-uchiwa/issues/14) +- Rename to puppet-uchiwa? [\#10](https://github.com/Yelp/puppet-uchiwa/issues/10) + +**Merged pull requests:** + +- remove unnecessary str2bool in install.pp since input is already validated [\#13](https://github.com/Yelp/puppet-uchiwa/pull/13) ([lreed](https://github.com/lreed)) + +## [0.2.5](https://github.com/Yelp/puppet-uchiwa/tree/0.2.5) (2014-09-11) +[Full Changelog](https://github.com/Yelp/puppet-uchiwa/compare/0.2.4...0.2.5) + +**Closed issues:** + +- Duplicate declaration apt [\#11](https://github.com/Yelp/puppet-uchiwa/issues/11) +- setting install\_repo to false breaks module [\#8](https://github.com/Yelp/puppet-uchiwa/issues/8) + +**Merged pull requests:** + +- use anchor pattern to fix containment of uchiwa class [\#12](https://github.com/Yelp/puppet-uchiwa/pull/12) ([LarsFronius](https://github.com/LarsFronius)) +- Fix for issue \#8 with package requiring repo [\#9](https://github.com/Yelp/puppet-uchiwa/pull/9) ([bjwschaap](https://github.com/bjwschaap)) + +## [0.2.4](https://github.com/Yelp/puppet-uchiwa/tree/0.2.4) (2014-08-11) +[Full Changelog](https://github.com/Yelp/puppet-uchiwa/compare/0.2.3...0.2.4) + +## [0.2.3](https://github.com/Yelp/puppet-uchiwa/tree/0.2.3) (2014-08-08) +[Full Changelog](https://github.com/Yelp/puppet-uchiwa/compare/0.2.2...0.2.3) + +## [0.2.2](https://github.com/Yelp/puppet-uchiwa/tree/0.2.2) (2014-08-08) +[Full Changelog](https://github.com/Yelp/puppet-uchiwa/compare/0.2.0...0.2.2) + +## [0.2.0](https://github.com/Yelp/puppet-uchiwa/tree/0.2.0) (2014-08-07) +[Full Changelog](https://github.com/Yelp/puppet-uchiwa/compare/0.1.1...0.2.0) + +**Merged pull requests:** + +- Fix template so it generates valid JSON [\#2](https://github.com/Yelp/puppet-uchiwa/pull/2) ([bodgit](https://github.com/bodgit)) +- Clean up puppet-lint errors [\#1](https://github.com/Yelp/puppet-uchiwa/pull/1) ([bodgit](https://github.com/bodgit)) + +## [0.1.1](https://github.com/Yelp/puppet-uchiwa/tree/0.1.1) (2014-08-05) + + +\* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)* \ No newline at end of file diff --git a/uchiwa/Gemfile b/uchiwa/Gemfile index 4609bd9b9..dc74f6855 100755 --- a/uchiwa/Gemfile +++ b/uchiwa/Gemfile @@ -13,6 +13,10 @@ group :development, :test do gem 'pry', :require => false gem 'simplecov', :require => false gem 'vagrant-wrapper', :require => false + gem "travis" + gem "travis-lint" + gem "puppet-blacksmith" + gem "guard-rake" end if facterversion = ENV['FACTER_GEM_VERSION'] @@ -27,4 +31,4 @@ else gem 'puppet', :require => false end -# vim:ft=ruby \ No newline at end of file +# vim:ft=ruby diff --git a/uchiwa/Gemfile.lock b/uchiwa/Gemfile.lock index 52c660624..d90674984 100644 --- a/uchiwa/Gemfile.lock +++ b/uchiwa/Gemfile.lock @@ -11,6 +11,7 @@ GEM aws-sdk (1.42.0) json (~> 1.4) nokogiri (>= 1.4.4) + backports (3.6.7) beaker (1.17.6) aws-sdk (= 1.42.0) blimpy (~> 0.6) @@ -43,12 +44,19 @@ GEM archive-tar-minitar excon (>= 0.38.0) json + domain_name (0.5.25) + unf (>= 0.0.5, < 1.0.0) + ethon (0.8.0) + ffi (>= 1.3.0) excon (0.39.5) extlib (0.9.16) facter (2.1.0) CFPropertyList (~> 2.2.6) faraday (0.9.0) multipart-post (>= 1.2, < 3) + faraday_middleware (0.10.0) + faraday (>= 0.7.4, < 0.10) + ffi (1.9.10) fission (0.5.0) CFPropertyList (~> 2.2) fog (1.23.0) @@ -75,6 +83,13 @@ GEM fog-core fog-json formatador (0.2.5) + gh (0.14.0) + addressable + backports + faraday (~> 0.8) + multi_json (~> 1.0) + net-http-persistent (>= 2.7) + net-http-pipeline google-api-client (0.7.1) addressable (>= 2.3.2) autoparse (>= 0.3.3) @@ -86,10 +101,24 @@ GEM retriable (>= 1.4) signet (>= 0.5.0) uuidtools (>= 2.1.0) + guard (2.13.0) + formatador (>= 0.2.4) + listen (>= 2.7, <= 4.0) + lumberjack (~> 1.0) + nenv (~> 0.1) + notiffany (~> 0.0) + pry (>= 0.9.12) + shellany (~> 0.0) + thor (>= 0.18.1) + guard-rake (1.0.0) + guard + rake hiera (1.3.4) json_pure highline (1.6.21) hocon (0.0.4) + http-cookie (1.0.2) + domain_name (~> 0.5) inflecto (0.0.2) inifile (2.0.2) ipaddress (0.8.0) @@ -98,6 +127,10 @@ GEM jwt (1.0.0) launchy (2.4.2) addressable (~> 2.3) + listen (3.0.5) + rb-fsevent (>= 0.9.3) + rb-inotify (>= 0.9) + lumberjack (1.0.9) metaclass (0.0.4) method_source (0.8.2) mime-types (1.25.1) @@ -106,10 +139,17 @@ GEM metaclass (~> 0.0.1) multi_json (1.10.1) multipart-post (2.0.0) + nenv (0.2.0) + net-http-persistent (2.9.4) + net-http-pipeline (1.0.1) net-scp (1.2.1) net-ssh (>= 2.6.5) net-ssh (2.9.1) + netrc (0.11.0) nokogiri (1.5.11) + notiffany (0.0.8) + nenv (~> 0.1) + shellany (~> 0.0) pry (0.10.1) coderay (~> 1.1.0) method_source (~> 0.8.1) @@ -119,6 +159,9 @@ GEM hiera (~> 1.0) json_pure rgen (~> 0.6.5) + puppet-blacksmith (3.3.1) + puppet (>= 2.7.16) + rest-client puppet-lint (0.3.2) puppet-syntax (1.3.0) rake @@ -129,11 +172,21 @@ GEM rake rspec rspec-puppet + pusher-client (0.6.2) + json + websocket (~> 1.0) rake (10.3.2) + rb-fsevent (0.9.6) + rb-inotify (0.9.5) + ffi (>= 0.5.0) rbvmomi (1.8.1) builder nokogiri (>= 1.4.1) trollop + rest-client (1.8.0) + http-cookie (>= 1.0.2, < 2.0) + mime-types (>= 1.16, < 3.0) + netrc (~> 0.7) retriable (1.4.1) rgen (0.6.6) rspec (2.99.0) @@ -155,6 +208,7 @@ GEM rspec (~> 2.99) rspec-its specinfra (~> 1.25) + shellany (0.0.1) signet (0.5.1) addressable (>= 2.2.3) faraday (>= 0.9.0.rc5) @@ -168,12 +222,28 @@ GEM slop (3.6.0) specinfra (1.25.8) thor (0.19.1) + travis (1.6.11) + addressable (~> 2.3) + backports + faraday (~> 0.9) + faraday_middleware (~> 0.9) + gh (~> 0.13) + highline (~> 1.6) + launchy (~> 2.1) + pry (~> 0.9) + pusher-client (~> 0.4) + typhoeus (~> 0.6, >= 0.6.8) + travis-lint (2.0.0) + json trollop (2.0) + typhoeus (0.8.0) + ethon (>= 0.8.0) unf (0.1.4) unf_ext unf_ext (0.0.6) uuidtools (2.1.5) vagrant-wrapper (1.2.1.1) + websocket (1.2.2) PLATFORMS ruby @@ -182,12 +252,19 @@ DEPENDENCIES beaker beaker-rspec (~> 2.2.4) facter + guard-rake pry puppet + puppet-blacksmith puppet-lint puppetlabs_spec_helper rake rspec-puppet serverspec simplecov + travis + travis-lint vagrant-wrapper + +BUNDLED WITH + 1.10.6 diff --git a/uchiwa/Rakefile b/uchiwa/Rakefile index 2ad6ca615..6b7d849f7 100755 --- a/uchiwa/Rakefile +++ b/uchiwa/Rakefile @@ -15,4 +15,11 @@ PuppetLint.configuration.send("disable_autoloader_layout") PuppetLint.configuration.send("disable_quoted_booleans") PuppetLint.configuration.send('disable_class_inherits_from_params_class') PuppetLint.configuration.ignore_paths = exclude_paths -PuppetSyntax.exclude_paths = exclude_paths \ No newline at end of file +PuppetSyntax.exclude_paths = exclude_paths + +# These gems aren't always present, for instance +# on Travis with --without development +begin + require 'puppet_blacksmith/rake_tasks' +rescue LoadError +end diff --git a/uchiwa/manifests/init.pp b/uchiwa/manifests/init.pp index f5bc008dd..7157b9e8a 100755 --- a/uchiwa/manifests/init.pp +++ b/uchiwa/manifests/init.pp @@ -44,7 +44,7 @@ # # [*repo_key_source*] # String -# Default: http://repos.sensuapp.org/apt/pubkey.gpg +# Default: http://repositories.sensuapp.org/apt/pubkey.gpg # GPG key for the repo we're installing # # [*manage_services*] @@ -95,6 +95,36 @@ # }] # An array of API endpoints to connect uchiwa to one or multiple sensu servers. # +# [*users*] +# Array of hashes +# An array of user credentials to access the uchiwa dashboard. If set, it takes +# precendence over 'user' and 'pass'. +# Example: +# ``` +# [{ +# 'username' => 'user1', +# 'password' => 'pass1', +# 'readonly' => false +# }, +# { +# 'username' => 'user2', +# 'password' => 'pass2', +# 'readonly' => true +# }] +# ``` +# +# [*auth*] +# Hash +# A hash containing the static public and private key paths for generating and +# validating JSON Web Token (JWT) signatures. +# Example: +# ``` +# { +# 'publickey' => '/path/to/uchiwa.rsa.pub', +# 'privatekey' => '/path/to/uchiwa.rsa' +# } +# ``` +# class uchiwa ( $package_name = $uchiwa::params::package_name, $service_name = $uchiwa::params::service_name, @@ -112,6 +142,8 @@ $pass = $uchiwa::params::pass, $refresh = $uchiwa::params::refresh, $sensu_api_endpoints = $uchiwa::params::sensu_api_endpoints, + $users = $uchiwa::params::users, + $auth = $uchiwa::params::auth ) inherits uchiwa::params { # validate parameters here @@ -131,6 +163,8 @@ validate_string($pass) validate_integer($refresh) validate_array($sensu_api_endpoints) + validate_array($users) + validate_hash($auth) anchor { 'uchiwa::begin': } -> class { 'uchiwa::install': } -> diff --git a/uchiwa/manifests/params.pp b/uchiwa/manifests/params.pp index 1683e081d..197dac79f 100755 --- a/uchiwa/manifests/params.pp +++ b/uchiwa/manifests/params.pp @@ -22,8 +22,8 @@ $install_repo = true $repo = 'main' $repo_source = undef - $repo_key_id = '8911D8FF37778F24B4E726A218609E3D7580C77F' - $repo_key_source = 'http://repos.sensuapp.org/apt/pubkey.gpg' + $repo_key_id = 'EE15CFF6AB6E4E290FDAB681A20F259AEB9C94BB' + $repo_key_source = 'http://repositories.sensuapp.org/apt/pubkey.gpg' $manage_services = true $manage_user = true @@ -45,4 +45,6 @@ $user = '' $pass = '' $refresh = '5' + $users = [] + $auth = {} } diff --git a/uchiwa/manifests/repo/apt.pp b/uchiwa/manifests/repo/apt.pp index eedd082df..a84cb8003 100755 --- a/uchiwa/manifests/repo/apt.pp +++ b/uchiwa/manifests/repo/apt.pp @@ -19,7 +19,7 @@ if $uchiwa::repo_source { $url = $uchiwa::repo_source } else { - $url = 'http://repos.sensuapp.org/apt' + $url = 'http://repositories.sensuapp.org/apt' } apt::source { 'sensu': diff --git a/uchiwa/metadata.json b/uchiwa/metadata.json index 5fe005e71..ae66d6fc9 100644 --- a/uchiwa/metadata.json +++ b/uchiwa/metadata.json @@ -56,7 +56,7 @@ } ], "name": "yelp-uchiwa", - "version": "0.3.0", + "version": "1.0.2", "author": "yelp", "summary": "Puppet module for installing Uchiwa", "license": "Apache 2.0", @@ -65,7 +65,12 @@ "issues_url": "https://github.com/yelp/puppet-uchiwa/issues", "description": "Puppet module for installing Uchiwa", "dependencies": [ - {"name":"puppetlabs/apt","version_requirement": ">=2.0.0 <3.0.0"}, - {"name":"puppetlabs/stdlib"} + { + "name": "puppetlabs/apt", + "version_requirement": ">=2.0.0 <3.0.0" + }, + { + "name": "puppetlabs/stdlib" + } ] } diff --git a/uchiwa/spec/classes/uchiwa_spec.rb b/uchiwa/spec/classes/uchiwa_spec.rb index 41608e7f8..d9e4ff7d5 100644 --- a/uchiwa/spec/classes/uchiwa_spec.rb +++ b/uchiwa/spec/classes/uchiwa_spec.rb @@ -53,11 +53,11 @@ context 'default' do it { should contain_apt__source('sensu').with( :ensure => 'present', - :location => 'http://repos.sensuapp.org/apt', + :location => 'http://repositories.sensuapp.org/apt', :release => 'sensu', :repos => 'main', :include => { 'src' => false, 'deb' => true }, - :key => { 'id' => '8911D8FF37778F24B4E726A218609E3D7580C77F', 'source' => 'http://repos.sensuapp.org/apt/pubkey.gpg' }, + :key => { 'id' => 'EE15CFF6AB6E4E290FDAB681A20F259AEB9C94BB', 'source' => 'http://repositories.sensuapp.org/apt/pubkey.gpg' }, :before => 'Package[uchiwa]' ) } end @@ -93,7 +93,7 @@ it { should_not contain_apt__key('sensu').with( :key => '7580C77F', - :key_source => 'http://repos.sensuapp.org/apt/pubkey.gpg' + :key_source => 'http://repositories.sensuapp.org/apt/pubkey.gpg' ) } it { should contain_package('uchiwa').with( @@ -152,4 +152,20 @@ } end + context 'with multiple users' do + let(:params) {{ :users => [ { 'username' => 'user1', 'password' => 'pass1', 'readonly' => true } ] }} + it { + should contain_file('/etc/sensu/uchiwa.json') \ + .with_content(/"username": "user1",\n "password": "pass1",\n "role": {\n "readonly": true\n }\n }/) + } + end + + context 'with static JWT RSA keys' do + let(:params) {{ :auth => { 'publickey' => '/etc/sensu/uchiwa.rsa.pub', 'privatekey' => '/etc/sensu/uchiwa.rsa' } }} + it { + should contain_file('/etc/sensu/uchiwa.json') \ + .with_content(/"auth": {\n "publickey": "\/etc\/sensu\/uchiwa.rsa.pub",\n "privatekey": "\/etc\/sensu\/uchiwa.rsa"\n }/) + } + end + end diff --git a/uchiwa/templates/etc/sensu/uchiwa.json.erb b/uchiwa/templates/etc/sensu/uchiwa.json.erb index 771f4a5fd..d2720fd52 100644 --- a/uchiwa/templates/etc/sensu/uchiwa.json.erb +++ b/uchiwa/templates/etc/sensu/uchiwa.json.erb @@ -20,6 +20,25 @@ "port": <%= @port %>, "user": "<%= @user %>", "pass": "<%= @pass %>", - "refresh": <%= @refresh %> + "refresh": <%= @refresh %><%= ',' if @users.size > 0 or @auth.size == 2 %> + <%- if @users.size > 0 -%> + "users": [ + <%- @users.each_with_index do |user, i| -%> + { + "username": "<%= user['username'] %>", + "password": "<%= user['password'] %>", + "role": { + "readonly": <%= user['readonly'] %> + } + }<%= ',' if i < (@users.size - 1) %> + <%- end -%> + ]<%= ',' if @auth.size == 2 %> + <%- end -%> + <%- if @auth.size == 2 -%> + "auth": { + "publickey": "<%= @auth['publickey'] %>", + "privatekey": "<%= @auth['privatekey'] %>" + } + <%- end -%> } } From e6b4afe0ddbdde168802c9b415b45402d87cf081 Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Wed, 16 Dec 2015 08:50:21 -0500 Subject: [PATCH 05/35] Remove fluentd module. This will be added again, but using a different upstream module. Change-Id: Ifabc26cc821fdd84f9d919de1bd1f81a5e30da32 (cherry picked from commit 671cbfbbbff7dda805e8a11772755be482458de2) --- Puppetfile | 4 - fluentd/.fixtures.yml | 8 - fluentd/.gemfile | 16 -- fluentd/.gitignore | 7 - fluentd/.travis.yml | 36 --- fluentd/CHANGELOG | 23 -- fluentd/Gemfile | 21 -- fluentd/LICENSE | 202 --------------- fluentd/Modulefile | 13 - fluentd/README.md | 195 --------------- fluentd/Rakefile | 2 - fluentd/files/packages.treasure-data.com.key | 18 -- .../lib/puppet/provider/package/fluentgem.rb | 137 ---------- fluentd/manifests/config.pp | 19 -- fluentd/manifests/configfile.pp | 15 -- fluentd/manifests/filter.pp | 35 --- fluentd/manifests/forest_match.pp | 15 -- fluentd/manifests/init.pp | 17 -- fluentd/manifests/install_plugin.pp | 40 --- fluentd/manifests/install_plugin/file.pp | 24 -- fluentd/manifests/install_plugin/gem.pp | 21 -- fluentd/manifests/install_repo/apt.pp | 24 -- fluentd/manifests/install_repo/yum.pp | 19 -- fluentd/manifests/match.pp | 17 -- fluentd/manifests/packages.pp | 50 ---- fluentd/manifests/params.pp | 12 - fluentd/manifests/service.pp | 12 - fluentd/manifests/source.pp | 17 -- fluentd/spec/classes/fluentd_config_spec.rb | 58 ----- fluentd/spec/classes/fluentd_packages_spec.rb | 70 ------ fluentd/spec/classes/fluentd_service_spec.rb | 79 ------ fluentd/spec/classes/fluentd_spec.rb | 233 ------------------ fluentd/spec/defines/fluent_filter_spec.rb | 39 --- .../spec/defines/fluentd_configfile_spec.rb | 30 --- fluentd/spec/defines/fluentd_match_spec.rb | 76 ------ .../fluentgem/gem-list-single-package | 4 - .../fluentgem/line-with-1.8.5-warning | 14 -- fluentd/spec/spec.opts | 4 - fluentd/spec/spec_helper.rb | 2 - fluentd/spec/unit/provider/fluentgem_spec.rb | 176 ------------- fluentd/spec_helper.rb | 1 - fluentd/templates/filter.erb | 30 --- fluentd/templates/forest_match.erb | 21 -- fluentd/templates/match.erb | 33 --- fluentd/templates/source.erb | 16 -- fluentd/templates/td-agent.conf.erb | 2 - 46 files changed, 1907 deletions(-) delete mode 100644 fluentd/.fixtures.yml delete mode 100644 fluentd/.gemfile delete mode 100644 fluentd/.gitignore delete mode 100644 fluentd/.travis.yml delete mode 100644 fluentd/CHANGELOG delete mode 100644 fluentd/Gemfile delete mode 100644 fluentd/LICENSE delete mode 100644 fluentd/Modulefile delete mode 100644 fluentd/README.md delete mode 100644 fluentd/Rakefile delete mode 100644 fluentd/files/packages.treasure-data.com.key delete mode 100644 fluentd/lib/puppet/provider/package/fluentgem.rb delete mode 100644 fluentd/manifests/config.pp delete mode 100644 fluentd/manifests/configfile.pp delete mode 100644 fluentd/manifests/filter.pp delete mode 100644 fluentd/manifests/forest_match.pp delete mode 100644 fluentd/manifests/init.pp delete mode 100644 fluentd/manifests/install_plugin.pp delete mode 100644 fluentd/manifests/install_plugin/file.pp delete mode 100644 fluentd/manifests/install_plugin/gem.pp delete mode 100644 fluentd/manifests/install_repo/apt.pp delete mode 100644 fluentd/manifests/install_repo/yum.pp delete mode 100644 fluentd/manifests/match.pp delete mode 100644 fluentd/manifests/packages.pp delete mode 100644 fluentd/manifests/params.pp delete mode 100644 fluentd/manifests/service.pp delete mode 100644 fluentd/manifests/source.pp delete mode 100644 fluentd/spec/classes/fluentd_config_spec.rb delete mode 100644 fluentd/spec/classes/fluentd_packages_spec.rb delete mode 100644 fluentd/spec/classes/fluentd_service_spec.rb delete mode 100644 fluentd/spec/classes/fluentd_spec.rb delete mode 100644 fluentd/spec/defines/fluent_filter_spec.rb delete mode 100644 fluentd/spec/defines/fluentd_configfile_spec.rb delete mode 100644 fluentd/spec/defines/fluentd_match_spec.rb delete mode 100644 fluentd/spec/fixtures/unit/provider/fluentgem/gem-list-single-package delete mode 100644 fluentd/spec/fixtures/unit/provider/fluentgem/line-with-1.8.5-warning delete mode 100644 fluentd/spec/spec.opts delete mode 100644 fluentd/spec/spec_helper.rb delete mode 100644 fluentd/spec/unit/provider/fluentgem_spec.rb delete mode 100644 fluentd/spec_helper.rb delete mode 100644 fluentd/templates/filter.erb delete mode 100644 fluentd/templates/forest_match.erb delete mode 100644 fluentd/templates/match.erb delete mode 100644 fluentd/templates/source.erb delete mode 100644 fluentd/templates/td-agent.conf.erb diff --git a/Puppetfile b/Puppetfile index 1db6e2d81..a27b08e9b 100644 --- a/Puppetfile +++ b/Puppetfile @@ -58,10 +58,6 @@ mod 'firewall', :commit => '1b6cc9192150d9521cc70301d0452daf189a63f7', :git => 'https://github.com/puppetlabs/puppetlabs-firewall.git' -mod 'fluentd', - :commit => '828fc99cd6c631e8b34f47b8fdcfa3d30dd8daa2', - :git => 'https://github.com/mmz-srf/puppet-fluentd.git' - mod 'galera', :commit => 'a2ecf273aef9ef9193bba35c235fb7b8b821a0c3', :git => 'https://github.com/redhat-openstack/puppet-galera.git' diff --git a/fluentd/.fixtures.yml b/fluentd/.fixtures.yml deleted file mode 100644 index 779d63a00..000000000 --- a/fluentd/.fixtures.yml +++ /dev/null @@ -1,8 +0,0 @@ -fixtures: - repositories: - apt: "https://github.com/puppetlabs/puppetlabs-apt.git" - concat: "https://github.com/puppetlabs/puppetlabs-concat.git" - stdlib: "https://github.com/puppetlabs/puppetlabs-stdlib.git" - symlinks: - "fluentd": "#{source_dir}" - diff --git a/fluentd/.gemfile b/fluentd/.gemfile deleted file mode 100644 index e7ea9dbb9..000000000 --- a/fluentd/.gemfile +++ /dev/null @@ -1,16 +0,0 @@ -source 'https://rubygems.org' - -group :development, :test do - gem 'rake', :require => false - gem 'rspec-puppet', :require => false - gem 'puppetlabs_spec_helper', :require => false - gem 'rspec-system', :require => false - gem 'rspec-system-puppet', :require => false - gem 'rspec-system-serverspec', :require => false -end - -if puppetversion = ENV['PUPPET_GEM_VERSION'] - gem 'puppet', puppetversion, :require => false -else - gem 'puppet', :require => false -end diff --git a/fluentd/.gitignore b/fluentd/.gitignore deleted file mode 100644 index e7beff00f..000000000 --- a/fluentd/.gitignore +++ /dev/null @@ -1,7 +0,0 @@ -pkg/ -Gemfile.lock -vendor/ -spec/fixtures/modules/ -.vagrant/ -.bundle/ -coverage/ diff --git a/fluentd/.travis.yml b/fluentd/.travis.yml deleted file mode 100644 index 0a8b188f4..000000000 --- a/fluentd/.travis.yml +++ /dev/null @@ -1,36 +0,0 @@ ---- -branches: - only: - - master -language: ruby -bundler_args: --without development -script: bundle exec rake spec SPEC_OPTS='--format documentation' -rvm: - - 1.9.3 - - 2.0.0 -gemfile: .gemfile -env: - matrix: - - PUPPET_GEM_VERSION="~> 2.7.0" - - PUPPET_GEM_VERSION="~> 3.1.0" - - PUPPET_GEM_VERSION="~> 3.2.0" - - PUPPET_GEM_VERSION="~> 3.3.0" - - PUPPET_GEM_VERSION="~> 3.4.0" - - PUPPET_GEM_VERSION="~> 3.5.0" - - PUPPET_GEM_VERSION="~> 3.6.0" -matrix: - exclude: - - rvm: 1.9.3 - env: PUPPET_GEM_VERSION="~> 2.7.0" - gemfile: .gemfile - - rvm: 1.9.3 - env: PUPPET_GEM_VERSION="~> 3.1.0" - gemfile: .gemfile - - rvm: 2.0.0 - env: PUPPET_GEM_VERSION="~> 2.7.0" - gemfile: .gemfile - - rvm: 2.0.0 - env: PUPPET_GEM_VERSION="~> 3.1.0" - gemfile: .gemfile -notifications: - email: gianni.carafa@srf.ch diff --git a/fluentd/CHANGELOG b/fluentd/CHANGELOG deleted file mode 100644 index 33cc56afb..000000000 --- a/fluentd/CHANGELOG +++ /dev/null @@ -1,23 +0,0 @@ -## puppet-fluentd changelog - -Release notes for the srfmmz/puppet-fluentd module. - ------------------------------------------- - -#### 0.1.4 - 2014/08/05 - -* FEATURE added filter functionality (by softecspa) -* FIX configfile syntax fix for td-agent v2 (by Molezeem) -* CHANGE changed the plugin installation to fluentgem provider (by saleksee) - -#### 0.1.3 - 2014/07/06 - -Please update to get the new repository of Treasuredata. - -Detailed Changes: - -* FEATURE : added time_format for source configuration -* CHANGE : Added new treasuredata repositories. - - for more info see here : https://groups.google.com/forum/#!topic/fluentd/46FnWIoLTrU -* CHANGE : Restart td-agent when config changed -* FEATURE : Startetd this changelog diff --git a/fluentd/Gemfile b/fluentd/Gemfile deleted file mode 100644 index e30fed0ea..000000000 --- a/fluentd/Gemfile +++ /dev/null @@ -1,21 +0,0 @@ -# Gemfile to run bundler command -# usagen: bundler exec rake spec - -source 'https://rubygems.org' - -puppetversion = ENV.key?('PUPPET_VERSION') ? "~> #{ENV['PUPPET_VERSION']}" : ['>= 3.2.1'] -gem 'puppet', puppetversion - -if puppetversion =~ /^3/ - ## rspec-hiera-puppet is puppet 3 only - gem 'rspec-hiera-puppet', '>=1.0.0' -end - -facterversion = ENV.key?('FACTER_VERSION') ? "~> #{ENV['FACTER_VERSION']}" : ['>= 1.7.1'] - -gem 'facter', facterversion - -gem 'rake' -gem 'rspec' -gem 'rspec-puppet', '>=0.1.6' -gem 'puppetlabs_spec_helper', '>=0.4.1' diff --git a/fluentd/LICENSE b/fluentd/LICENSE deleted file mode 100644 index e06d20818..000000000 --- a/fluentd/LICENSE +++ /dev/null @@ -1,202 +0,0 @@ -Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "{}" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright {yyyy} {name of copyright owner} - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - diff --git a/fluentd/Modulefile b/fluentd/Modulefile deleted file mode 100644 index 520e5e410..000000000 --- a/fluentd/Modulefile +++ /dev/null @@ -1,13 +0,0 @@ -name 'srf-fluentd' -version '0.1.4' - -dependency 'puppetlabs/apt', '>= 1.4.0' -dependency 'puppetlabs/concat', '>= 1.0.0' -dependency 'puppetlabs/stdlib', '>= 4.1.0' - -summary 'fluentd module' -description 'Manages Fluentd installation and configuration with Puppet using the td-agent.' -source 'https://github.com/mmz-srf/puppet-fluentd.git' -project_page 'https://github.com/mmz-srf/puppet-fluentd' -author 'Gianni Carafa' -license 'Apache License, Version 2.0' diff --git a/fluentd/README.md b/fluentd/README.md deleted file mode 100644 index f46c78617..000000000 --- a/fluentd/README.md +++ /dev/null @@ -1,195 +0,0 @@ -puppet-fluentd -============== - -[![Build Status](https://travis-ci.org/mmz-srf/puppet-fluentd.png?branch=master)](https://travis-ci.org/mmz-srf/puppet-fluentd) - -Manage Fluentd installation, configuration and Plugin-management with Puppet using the td-agent. - -## Supported Operating Systems -- Debian (tested on Debian 7.5) -- Ubuntu -- Redhat -- CentOS (tested on CentOS 6.4) - -## Used Modules -- apt: "https://github.com/puppetlabs/puppetlabs-apt.git" (Only for Debian) -- concat: "https://github.com/puppetlabs/puppetlabs-concat.git" -- stdlib: "https://github.com/puppetlabs/puppetlabs-stdlib.git" - -## Contributing -* Fork it -* Create a feature branch (`git checkout -b my-new-feature`) -* Run rspec tests (`bundle exec rake spec` or `rake spec`) -* Commit your changes (`git commit -am 'Added some feature'`) -* Push to the branch (`git push origin my-new-feature`) -* Create new Pull Request - -### Branches in this module -- **master** tested and working. Latest Version on http://forge.puppetlabs.com -- **developmemt** includes the newewst features. works too, mostly. - -## Configuration -How to configure a Agent to send data to a centralised Fluentd-Server - -### Install a Plugin -Install your fluentd plugin. (check here for the right pluginname : http://fluentd.org/plugin/ ). - -You can choose from a file or gem based instalation. -``` - include ::fluentd - - fluentd::install_plugin { 'elasticsearch': - plugin_type => 'gem', - plugin_name => 'fluent-plugin-elasticsearch', - } -``` - -### Create a Agent -The Agent watches over your logfiles and sends its content to the Collector. -``` - include ::fluentd - - fluentd::configfile { 'apache': } - fluentd::source { 'apache_main': - configfile => 'apache' - type => 'tail', - format => 'apache2', - tag => 'apache.access_log', - config => { - 'path' => '/var/log/apache2/access.log', - 'pos_file' => '/var/tmp/fluentd.pos', - }, - notify => Class['fluentd::service'] - } - - fluentd::configfile { 'syslog': } - fluentd::source { 'syslog_main': - configfile => 'syslog', - type => 'tail', - format => 'syslog', - tag => 'system.syslog', - config => { - 'path' => '/var/log/syslog', - 'pos_file' => '/tmp/td-agent.syslog.pos', - }, - notify => Class['fluentd::service'] - } - - fluentd::configfile { 'forward': } - fluentd::match { 'forward_main': - configfile => 'forward' - pattern => '**', - type => 'forward', - servers => [ - {'host' => 'PUT_YOUR_HOST_HERE', 'port' => '24224'} - ], - notify => Class['fluentd::service'] - } -``` -#### creates on the Agent side following files : -``` -/etc/td-agent/ - ├── config.d - │   ├── apache.conf - │   ├── syslog.conf - │   └── forward.conf - ├── ... - ... -``` - -### Create a Collector -The Collector collects all the data from the Agents. He now stores the data in files, Elasticsearch or elsewhere. -``` - include ::fluentd - - fluentd::configfile { 'collector': } - fluentd::source { 'collector_main': - configfile => 'collector', - type => 'forward', - notify => Class['fluentd::service'] - } - - fluentd::match { 'collector_main': - configfile => 'collector', - pattern => '**', - type => 'elasticsearch', - config => { - 'logstash_format' => true, - }, - notify => Class['fluentd::service'] - } - - # receive syslog messages on port 5140 - # all rsyslog daemons on the clients sends theire messages to 5140 - fluentd::configfile { 'rsyslog': } - fluentd::source { 'rsyslog_main': - configfile => 'rsyslog', - type => 'syslog', - tag => 'system.local', - config => { - 'port' => '5140', - 'bind' => '0.0.0.0', - }, - notify => Class['fluentd::service'] - } -``` - -#### creates on the Collectors side following files : -``` -/etc/td-agent/ - ├── config.d - │   └── collector.conf - ├── ... - ... -``` - -### copy ouput to multiple stores -```` - $logger=[ { 'host' => 'logger-sample01', 'port' => '24224'}, - { 'host' => 'logger-example01', 'port' => '24224', 'standby' => ''} ] - - fluentd::match { 'forward_to_logger': - configfile => 'sample_tail', - pattern => 'alocal', - type => 'copy', - config => [ - { - 'type' => 'forward', - 'send_timeout' => '60s', - 'recover_wait' => '10s', - 'heartbeat_interval' => '1s', - 'phi_threshold' => 8, - 'hard_timeout' => '60s', - 'flush_interval' => '5s', - 'servers' => $logger, - }, - { - 'type' => 'stdout', - 'output_type' => 'json', - } - ], - notify => Class['fluentd::service'] - } - -``` - -### add a filter -``` - fluentd::configfile { 'myfilter': } - fluentd::filter { 'myfilter_main': - configfile => 'myfilter', - pattern => '**', - type => 'grep', - input_key => 'key', - regexp => '/*.foo.*/', - exclude => 'baar', - output_tag => 'mytag', - add_tag_prefix => 'pre_', - remove_tag_prefix => 'remove_', - add_tag_suffix => '_after', - remove_tag_suffix => '_remove', - config => { - 'customvalue' => true, - } - } -``` diff --git a/fluentd/Rakefile b/fluentd/Rakefile deleted file mode 100644 index bbfcc8b9c..000000000 --- a/fluentd/Rakefile +++ /dev/null @@ -1,2 +0,0 @@ -require 'rubygems' -require 'puppetlabs_spec_helper/rake_tasks' \ No newline at end of file diff --git a/fluentd/files/packages.treasure-data.com.key b/fluentd/files/packages.treasure-data.com.key deleted file mode 100644 index c86c35489..000000000 --- a/fluentd/files/packages.treasure-data.com.key +++ /dev/null @@ -1,18 +0,0 @@ ------BEGIN PGP PUBLIC KEY BLOCK----- -Version: GnuPG v1.4.5 (GNU/Linux) - -mQGiBFKuy6MRBACJccrkWHfYSKSbchD4ywYLGkIKITSLzEXrlpy4Sp6Mr9G1OGIv -buIUIxksnev6h6wlFgbFYTLngpod+Jn4DpcGgXqzZ3lhOY1fvAaQ4SJ4RUELm1JJ -SEWE0uYHuoI4+TrLtod3U02ETD+Lf6LWO3IfU1AYsA467tDYpVXR0GHeAwCgqU/V -M+ypNOYFHVs1/aX83wdg0K8D+wf+tcvTHb/i2lND5DVfTWtC54WYNIG6QO8DgGdu -EYregcMKRRNM4nSfUISzCgrjnpLGjvB9NKONCrimDiuz4TH7KDcEsKGA+zTzJLCy -mcE6tKEHV5yS7cK0inim4nxAKFB70306CkhHyL2TrB42ppNFJwG1igO4BJ+55b9X -Dd03A/9o6ONFle6khG+UE82WYTayy2pAshKTSLnOlB7CIxwt3SrzFXTuNosajVgZ -S0EIB8c6SjFN6P1jHjd7PYOmBxd2xptERTg8CJMoTUI+P7LypWuvmagIdEwe571d -ZMQ26HNuHg7/IMg+FQxhl/DFU6Aq+2CBJ3q8gO8Us4/UYc1PhrRUVHJlYXN1cmUg -RGF0YSwgSW5jIChUcmVhc3VyZSBBZ2VudCBPZmZpY2lhbCBTaWduaW5nIGtleSkg -PHN1cHBvcnRAdHJlYXN1cmUtZGF0YS5jb20+iGAEExECACAFAlKuy6MCGwMGCwkI -BwMCBBUCCAMEFgIDAQIeAQIXgAAKCRAQk9tFoS4gb3HgAKCBCDNT/cWzkKRQxvnL -LiGvPetVSACeLLKnt2twfiPc9ZEA/X8MKkPjWqc= -=e5Fe ------END PGP PUBLIC KEY BLOCK----- diff --git a/fluentd/lib/puppet/provider/package/fluentgem.rb b/fluentd/lib/puppet/provider/package/fluentgem.rb deleted file mode 100644 index 9c5165481..000000000 --- a/fluentd/lib/puppet/provider/package/fluentgem.rb +++ /dev/null @@ -1,137 +0,0 @@ -require 'puppet/provider/package' -require 'uri' - -# Ruby gems support. -Puppet::Type.type(:package).provide :fluentgem, :parent => Puppet::Provider::Package do - desc "Install gem via fluent-gem (included by td-agent). Ruby Gem support. If a URL is passed via `source`, then that URL is used as the - remote gem repository; if a source is present but is not a valid URL, it will be - interpreted as the path to a local gem file. If source is not present at all, - the gem will be installed from the default gem repositories. - - This provider supports the `install_options` attribute, which allows command-line flags to be passed to the gem command. - These options should be specified as a string (e.g. '--flag'), a hash (e.g. {'--flag' => 'value'}), - or an array where each element is either a string or a hash." - - has_feature :versionable, :install_options - - ENV['PATH'] = "#{ENV['PATH']}:/usr/lib64/fluent/ruby/bin:/usr/lib/fluent/ruby/bin" - - commands :gemcmd => "fluent-gem" - - def self.gemlist(options) - gem_list_command = [command(:gemcmd), "list"] - - if options[:local] - gem_list_command << "--local" - else - gem_list_command << "--remote" - end - if options[:source] - gem_list_command << "--source" << options[:source] - end - if name = options[:justme] - gem_list_command << "^" + name + "$" - end - - begin - list = execute(gem_list_command).lines. - map {|set| gemsplit(set) }. - reject {|x| x.nil? } - rescue Puppet::ExecutionFailure => detail - raise Puppet::Error, "Could not list gems: #{detail}", detail.backtrace - end - - if options[:justme] - return list.shift - else - return list - end - end - - def self.gemsplit(desc) - # `gem list` when output console has a line like: - # *** LOCAL GEMS *** - # but when it's not to the console that line - # and all blank lines are stripped - # so we don't need to check for them - - if desc =~ /^(\S+)\s+\((.+)\)/ - name = $1 - versions = $2.split(/,\s*/) - { - :name => name, - :ensure => versions.map{|v| v.split[0]}, - :provider => :gem - } - else - Puppet.warning "Could not match #{desc}" unless desc.chomp.empty? - nil - end - end - - def self.instances(justme = false) - gemlist(:local => true).collect do |hash| - new(hash) - end - end - - def install(useversion = true) - command = [command(:gemcmd), "install"] - command << "-v" << resource[:ensure] if (! resource[:ensure].is_a? Symbol) and useversion - - if source = resource[:source] - begin - uri = URI.parse(source) - rescue => detail - self.fail Puppet::Error, "Invalid source '#{uri}': #{detail}", detail - end - - case uri.scheme - when nil - # no URI scheme => interpret the source as a local file - command << source - when /file/i - command << uri.path - when 'puppet' - # we don't support puppet:// URLs (yet) - raise Puppet::Error.new("puppet:// URLs are not supported as gem sources") - else - # interpret it as a gem repository - command << "--source" << "#{source}" << resource[:name] - end - else - command << "--no-rdoc" << "--no-ri" << resource[:name] - end - - command += install_options if resource[:install_options] - - output = execute(command) - # Apparently some stupid gem versions don't exit non-0 on failure - self.fail "Could not install: #{output.chomp}" if output.include?("ERROR") - end - - def latest - # This always gets the latest version available. - gemlist_options = {:justme => resource[:name]} - gemlist_options.merge!({:source => resource[:source]}) unless resource[:source].nil? - hash = self.class.gemlist(gemlist_options) - - hash[:ensure][0] - end - - def query - self.class.gemlist(:justme => resource[:name], :local => true) - end - - def uninstall - gemcmd "uninstall", "-x", "-a", resource[:name] - end - - def update - self.install(false) - end - - def install_options - join_options(resource[:install_options]) - end -end diff --git a/fluentd/manifests/config.pp b/fluentd/manifests/config.pp deleted file mode 100644 index f9d62526c..000000000 --- a/fluentd/manifests/config.pp +++ /dev/null @@ -1,19 +0,0 @@ -# Class: fluentd::config() -# -# -class fluentd::config() { - file { '/etc/td-agent/td-agent.conf' : - ensure => file, - owner => 'root', - group => 'root', - content => template('fluentd/td-agent.conf.erb'), - notify => Class['fluentd::service'], - } - - file {'/etc/td-agent/config.d': - ensure => 'directory', - owner => 'td-agent', - group => 'td-agent', - mode => '0750', - } -} \ No newline at end of file diff --git a/fluentd/manifests/configfile.pp b/fluentd/manifests/configfile.pp deleted file mode 100644 index 509cf4eef..000000000 --- a/fluentd/manifests/configfile.pp +++ /dev/null @@ -1,15 +0,0 @@ -# == definition fluentd::configfile -define fluentd::configfile { - $source_conf = "/etc/td-agent/config.d/${title}.conf" - if ! defined(Class['fluentd']) { - fail('You must include the fluentd base class before using any fluentd defined resources') - } - concat{$source_conf: - owner => 'td-agent', - group => 'td-agent', - mode => '0644', - require => Class['Fluentd::Packages'], - notify => Class['Fluentd::Service'], - } -} - diff --git a/fluentd/manifests/filter.pp b/fluentd/manifests/filter.pp deleted file mode 100644 index 1d55517d7..000000000 --- a/fluentd/manifests/filter.pp +++ /dev/null @@ -1,35 +0,0 @@ -# == definition fluentd::filter -define fluentd::filter ( - $configfile, - $pattern, - $type = 'grep', - $input_key = '', - $regexp = '', - $exclude = '', - $config = {}, - $output_tag = '', - $add_tag_prefix = '', - $remove_tag_prefix = '', - $add_tag_suffix = '', - $remove_tag_suffix = '', -) { - - if ($type == 'grep') { - if !defined(Fluentd::Install_plugin['fluent-plugin-grep']) { - fluentd::install_plugin{'fluent-plugin-grep': - plugin_type => 'gem' - } - } - } - - if ((($regexp != '') or ($exclude != '')) and ($input_key == '')) or (($input_key != '') and ($regexp == '') and ($exclude == '')) { - fail ('regexp, exlude and input_key must be used in conjuction') - } - - concat::fragment { 'filter': - target => "/etc/td-agent/config.d/${configfile}.conf", - require => Class['Fluentd::Packages'], - content => template('fluentd/filter.erb'), - } - -} diff --git a/fluentd/manifests/forest_match.pp b/fluentd/manifests/forest_match.pp deleted file mode 100644 index 09ca9a996..000000000 --- a/fluentd/manifests/forest_match.pp +++ /dev/null @@ -1,15 +0,0 @@ -# == definition fluentd::forest_match -define fluentd::forest_match ( - $configfile, - $type, - $pattern, - $config = {}, - $servers = [], -) { - - concat::fragment { "match_${title}": - target => "/etc/td-agent/config.d/${configfile}.conf", - require => Package["${fluentd::package_name}"], - content => template('fluentd/forest_match.erb'), - } -} diff --git a/fluentd/manifests/init.pp b/fluentd/manifests/init.pp deleted file mode 100644 index f0e095634..000000000 --- a/fluentd/manifests/init.pp +++ /dev/null @@ -1,17 +0,0 @@ -# == class fluentd -class fluentd ( - $version = '1', - $package_name = $fluentd::params::package_name, - $install_repo = $fluentd::params::install_repo, - $package_ensure = $fluentd::params::package_ensure, - $service_enable = $fluentd::params::service_enable, - $service_ensure = $fluentd::params::service_ensure -) inherits fluentd::params { - class{'fluentd::packages': } - class{'fluentd::config': } - class{'fluentd::service': } - - validate_bool($install_repo, $service_enable) - - Class['Fluentd::Packages'] -> Class['Fluentd::Config'] -> Class['Fluentd::Service'] -} diff --git a/fluentd/manifests/install_plugin.pp b/fluentd/manifests/install_plugin.pp deleted file mode 100644 index a395b7fde..000000000 --- a/fluentd/manifests/install_plugin.pp +++ /dev/null @@ -1,40 +0,0 @@ -# == fluentd::install_plugin -# -# install a plugin with either /usr/lib/fluent/ruby/bin/fluent-gem or a file -# -# you need to sepecify wich one of theese options plus the name. -# -# Parameters: -# the name of this ressource reflects the either the filename of the plugin, which must -# be copied to fluentd/files or the name of the gem -# -# plugin_type: specify "file" to copy that file to /etc/fluentd/plugins -# specify "gem" to try to install the plugin with fluent-gem -# -# ensure: "present"(default) or "absent", install or uninstall a plugin -# -define fluentd::install_plugin ( - $plugin_type, - $ensure = 'present', - $plugin_name = $name, -) { - case $plugin_type { - 'file': { - fluentd::install_plugin::file { - [$plugin_name]: - ensure => $ensure, - require => Class['Fluentd::Packages'] - } - } - 'gem': { - fluentd::install_plugin::gem { - [$plugin_name]: - ensure => $ensure, - require => Class['Fluentd::Packages'] - } - } - default: { - fail("plugin_type => ${plugin_type} is currently not supportet by this module") - } - } -} diff --git a/fluentd/manifests/install_plugin/file.pp b/fluentd/manifests/install_plugin/file.pp deleted file mode 100644 index 47d3ab155..000000000 --- a/fluentd/manifests/install_plugin/file.pp +++ /dev/null @@ -1,24 +0,0 @@ -# == fluentd::install_plugin::file -# -# install a plugin with by copying a file to /etc/td-agent/plugins -# -# Parameters: -# the name of this ressource reflects the filename of the plugin, which must -# be copied to fluentd/files -# -# ensure: "present"(default) or "absent", install or uninstall a plugin -# -define fluentd::install_plugin::file ( - $ensure = 'present', - $plugin_name = $name, -) { - file { - "/etc/td-agent/plugin/${plugin_name}": - ensure => $ensure, - owner => td-agent, - group => td-agent, - mode => '0640', - source => "puppet:///fluentd/plugins/${plugin_name}", - notify => Service["${::fluentd::service_name}"]; - } -} diff --git a/fluentd/manifests/install_plugin/gem.pp b/fluentd/manifests/install_plugin/gem.pp deleted file mode 100644 index bd8e14508..000000000 --- a/fluentd/manifests/install_plugin/gem.pp +++ /dev/null @@ -1,21 +0,0 @@ -# == fluentd::install_plugin::gem -# -# install a plugin with /usr/lib/fluent/ruby/bin/fluent-gem -# -# Parameters: -# the name of this ressource reflects the name of the gem -# -# ensure: "present"(default) or "absent", install or uninstall a plugin -# -define fluentd::install_plugin::gem ( - $ensure = 'latest', - $plugin_name = $name, -) { - - package { $plugin_name: - ensure => $ensure, - provider => 'fluentgem', - notify => Service["${fluentd::service_name}"]; - } - -} diff --git a/fluentd/manifests/install_repo/apt.pp b/fluentd/manifests/install_repo/apt.pp deleted file mode 100644 index e2eadbfc1..000000000 --- a/fluentd/manifests/install_repo/apt.pp +++ /dev/null @@ -1,24 +0,0 @@ -##apt.pp - -# Class: fluentd::install_repo::apt () -# -# -class fluentd::install_repo::apt () { - - apt::source { 'treasure-data': - location => "http://packages.treasuredata.com/debian", - release => "lucid", - repos => "contrib", - include_src => false, - } - - file { '/tmp/packages.treasure-data.com.key': - ensure => file, - source => 'puppet:///modules/fluentd/packages.treasure-data.com.key' - }-> - exec { "import gpg key Treasure Data": - command => "/bin/cat /tmp/packages.treasure-data.com.key | apt-key add -", - unless => "/usr/bin/apt-key list | grep -q 'Treasure Data'", - notify => Class['::apt::update'], - } -} diff --git a/fluentd/manifests/install_repo/yum.pp b/fluentd/manifests/install_repo/yum.pp deleted file mode 100644 index 790781465..000000000 --- a/fluentd/manifests/install_repo/yum.pp +++ /dev/null @@ -1,19 +0,0 @@ -# yum.pp - -# Class: fluentd::install_repo::yum () -# -# -class fluentd::install_repo::yum ( - $key = $fluentd::params::yum_key_url, - ) { - - # Sorry for the different naming of the Repository between debian and redhat. - # But I dont want rename it to avoid a duplication. - yumrepo { 'treasuredata': - descr => 'Treasure Data', - baseurl => 'http://packages.treasuredata.com/redhat/$basearch', - gpgkey => 'http://packages.treasuredata.com/redhat/RPM-GPG-KEY-td-agent', - gpgcheck => 1, - } - -} diff --git a/fluentd/manifests/match.pp b/fluentd/manifests/match.pp deleted file mode 100644 index 49b3f2d93..000000000 --- a/fluentd/manifests/match.pp +++ /dev/null @@ -1,17 +0,0 @@ -# == definition fluentd::match -define fluentd::match ( - $configfile, - $type, - $pattern, - $config = {}, - $servers = [], - $order = undef, -) { - - concat::fragment { "match_${title}": - target => "/etc/td-agent/config.d/${configfile}.conf", - require => Package["${fluentd::package_name}"], - content => template('fluentd/match.erb'), - order => $order, - } -} diff --git a/fluentd/manifests/packages.pp b/fluentd/manifests/packages.pp deleted file mode 100644 index e48793579..000000000 --- a/fluentd/manifests/packages.pp +++ /dev/null @@ -1,50 +0,0 @@ -# == class fluentd::packages -class fluentd::packages ( - $package_name = $fluentd::package_name, - $install_repo = $fluentd::install_repo, - $package_ensure = $fluentd::package_ensure -){ - if $install_repo { - case $::osfamily { - 'redhat': { - class{'fluentd::install_repo::yum': - before => Package[$package_name], - } - } - 'debian': { - class{'fluentd::install_repo::apt': - before => Package[$package_name], - } - } - default: { - fail("Unsupported osfamily ${::osfamily}") - } - } - } - package { "$package_name": - ensure => $package_ensure - } - -# extra bits... why this is required isn't quite clear. - case $::osfamily { - 'debian': { - package{[ - 'libxslt1.1', - 'libyaml-0-2', - ]: - before => Package[$package_name], - ensure => $package_ensure - } - exec {'add user td-agent to group adm': - provider => shell, - unless => '/bin/grep -q "adm\S*td-agent" /etc/group', - command => '/usr/sbin/usermod -aG adm td-agent', - subscribe => Package[$package_name], - } - } - default: { - info("No required fluentd::packages extra bits for ${::osfamily}") - } - } - -} diff --git a/fluentd/manifests/params.pp b/fluentd/manifests/params.pp deleted file mode 100644 index aed3b9a31..000000000 --- a/fluentd/manifests/params.pp +++ /dev/null @@ -1,12 +0,0 @@ -# Class: fluentd::params -# -# -class fluentd::params { - $package_name = 'td-agent' - $package_ensure = 'installed' - $install_repo = true - $service_ensure = 'running' - $service_enable = true - $service_name = 'td-agent' - $yum_key_url = "http://packages.treasure-data.com/redhat/RPM-GPG-KEY-td-agent" -} \ No newline at end of file diff --git a/fluentd/manifests/service.pp b/fluentd/manifests/service.pp deleted file mode 100644 index a539f0978..000000000 --- a/fluentd/manifests/service.pp +++ /dev/null @@ -1,12 +0,0 @@ -# == class fluentd::service -class fluentd::service ( - $service_ensure = $fluentd::service_ensure, - $service_enable = $fluentd::service_enable, -) { - include fluentd::params - service {"${fluentd::params::service_name}": - ensure => $service_ensure, - enable => $service_enable, - hasstatus => true - } -} diff --git a/fluentd/manifests/source.pp b/fluentd/manifests/source.pp deleted file mode 100644 index b71738474..000000000 --- a/fluentd/manifests/source.pp +++ /dev/null @@ -1,17 +0,0 @@ -# == definition fluentd::source -define fluentd::source ( - $configfile, - $type, - $tag = false, - $format = false, - $time_format = false, - $config = {}, - $pattern = [], -) { - - concat::fragment { "source_${title}": - target => "/etc/td-agent/config.d/${configfile}.conf", - require => Package["${fluentd::package_name}"], - content => template('fluentd/source.erb'), - } -} diff --git a/fluentd/spec/classes/fluentd_config_spec.rb b/fluentd/spec/classes/fluentd_config_spec.rb deleted file mode 100644 index cb744b569..000000000 --- a/fluentd/spec/classes/fluentd_config_spec.rb +++ /dev/null @@ -1,58 +0,0 @@ -#!/usr/bin/env rspec -require 'spec_helper' - -describe 'fluentd::config', :type => :class do - context "On a Debian OS, config files should be written" do - let :facts do - { - :osfamily => 'Debian', - :operatingsystemrelease => '7', - :concat_basedir => '/tmp', - :lsbdistid => 'Debian', - } - end - - it "/etc/td-agent/td-agent.conf should be in place" do - should contain_file("/etc/td-agent/td-agent.conf").with( - 'ensure' => 'file', - 'owner' => 'root', - 'group' => 'root', - 'notify' => 'Class[Fluentd::Service]' - ).with_content(/^# Include.*config.d.*/m) - end - it "/etc/td-agent/config.d is created" do - should contain_file("/etc/td-agent/config.d").with( - 'ensure' => 'directory', - 'owner' => 'td-agent', - 'group' => 'td-agent', - 'mode' => '0750' - ) - end - end - context "On a Redhat OS, config files should be written" do - let :facts do - { - :osfamily => 'RedHat', - :operatingsystemrelease => '6.5', - :concat_basedir => '/tmp', - :lsbdistid => 'CentOS', - } - end - it "/etc/td-agent/td-agent.conf should be in place" do - should contain_file("/etc/td-agent/td-agent.conf").with( - 'ensure' => 'file', - 'owner' => 'root', - 'group' => 'root', - 'notify' => 'Class[Fluentd::Service]' - ).with_content(/^# Include.*config.d.*/m) - end - it "/etc/td-agent/config.d is created" do - should contain_file("/etc/td-agent/config.d").with( - 'ensure' => 'directory', - 'owner' => 'td-agent', - 'group' => 'td-agent', - 'mode' => '0750' - ) - end - end -end diff --git a/fluentd/spec/classes/fluentd_packages_spec.rb b/fluentd/spec/classes/fluentd_packages_spec.rb deleted file mode 100644 index bad912abb..000000000 --- a/fluentd/spec/classes/fluentd_packages_spec.rb +++ /dev/null @@ -1,70 +0,0 @@ -#!/usr/bin/env rspec -require 'spec_helper' - -describe 'fluentd::packages', :type => :class do - let (:params) {{:package_name => 'td-agent', :package_ensure => 'installed'}} - context "On a Debian OS" do - let :facts do - { - :osfamily => 'Debian', - :operatingsystemrelease => '7', - :lsbdistid => 'Debian', - } - end - - context "with install_repo=>true" do - let(:params) { {:install_repo => true} } - it do - should contain_apt__source("treasure-data").with( - 'location' => 'http://packages.treasuredata.com/debian' - ) - end - end - it { should contain_package("libxslt1.1").with( - 'ensure' => 'installed' - ) - } - it { should contain_package("libyaml-0-2").with( - 'ensure' => 'installed' - ) - } - it { should contain_package("td-agent").with( - 'ensure' => 'installed' - ) - } - end - - context "On a RedHat/Centos OS" do - - let :params do - { - :package_name => 'td-agent', - :package_ensure => 'running', - } - end - - let :facts do - { - :osfamily => 'Redhat', - } - end - - it { should contain_class('fluentd::packages')} - - context "with install_repo=>true" do - let(:params) { {:install_repo => true} } - it do - should contain_yumrepo('treasuredata').with( - 'baseurl' => 'http://packages.treasuredata.com/redhat/$basearch', - 'gpgkey' => 'http://packages.treasuredata.com/redhat/RPM-GPG-KEY-td-agent', - 'gpgcheck' => 1 - ) - end - end - - it { should contain_package("td-agent").with( - 'ensure' => 'running' - ) - } - end -end diff --git a/fluentd/spec/classes/fluentd_service_spec.rb b/fluentd/spec/classes/fluentd_service_spec.rb deleted file mode 100644 index c9ec2bbaf..000000000 --- a/fluentd/spec/classes/fluentd_service_spec.rb +++ /dev/null @@ -1,79 +0,0 @@ -#!/usr/bin/env rspec -require 'spec_helper' - -describe 'fluentd::service', :type => :class do - context "On a Debian OS" do - let :facts do - { - :osfamily => 'Debian', - :operatingsystemrelease => '7', - :concat_basedir => '/tmp', - :lsbdistid => 'Debian', - } - end - context "td-agent running and enabled" do - let (:params) {{:service_ensure => 'running', :service_enable => true}} - it { should contain_service("td-agent").with( - 'ensure' => 'running', - 'enable' => 'true', - 'hasstatus' => 'true' - ) - } - end - context "td-agent stopped but enabled" do - let (:params) {{:service_ensure => 'stopped', :service_enable => true}} - it { should contain_service("td-agent").with( - 'ensure' => 'stopped', - 'enable' => 'true', - 'hasstatus' => 'true' - ) - } - end - context "td-agent stopped and disabled" do - let (:params) {{:service_ensure => 'stopped', :service_enable => false}} - it { should contain_service("td-agent").with( - 'ensure' => 'stopped', - 'enable' => 'false', - 'hasstatus' => 'true' - ) - } - end - end - context "On a Redhat OS" do - let :facts do - { - :osfamily => 'RedHat', - :operatingsystemrelease => '6.5', - :concat_basedir => '/tmp', - :lsbdistid => 'CentOs', - } - end - context "td-agent running and enabled" do - let (:params) {{:service_ensure => 'running', :service_enable => true}} - it { should contain_service("td-agent").with( - 'ensure' => 'running', - 'enable' => 'true', - 'hasstatus' => 'true' - ) - } - end - context "td-agent stopped but enabled" do - let (:params) {{:service_ensure => 'stopped', :service_enable => true}} - it { should contain_service("td-agent").with( - 'ensure' => 'stopped', - 'enable' => 'true', - 'hasstatus' => 'true' - ) - } - end - context "td-agent stopped and disabled" do - let (:params) {{:service_ensure => 'stopped', :service_enable => false}} - it { should contain_service("td-agent").with( - 'ensure' => 'stopped', - 'enable' => 'false', - 'hasstatus' => 'true' - ) - } - end - end -end diff --git a/fluentd/spec/classes/fluentd_spec.rb b/fluentd/spec/classes/fluentd_spec.rb deleted file mode 100644 index 0d5c7cb4d..000000000 --- a/fluentd/spec/classes/fluentd_spec.rb +++ /dev/null @@ -1,233 +0,0 @@ -#!/usr/bin/env rspec -require 'spec_helper' - -describe 'fluentd', :type => :class do - context "On a Debian OS " do - let :facts do - { - :osfamily => 'Debian', - :operatingsystemrelease => '7', - :concat_basedir => '/tmp', - :lsbdistid => 'Debian', - } - end - context "Debian with repo installed" do - let (:params) {{ - :package_name => 'td-agent', - :package_ensure => 'installed', - :install_repo => true, - :service_enable => true, - :service_ensure => 'running' - }} - it "has fluentd::packages" do - should contain_class('fluentd::packages') - end - it "has fluentd::config class" do - should contain_class('fluentd::config') - end - it "has fluentd::service class" do - should contain_class('fluentd::service') - end - it "the apt repo file exists" do - should contain_apt__source("treasure-data").with( - 'location' => 'http://packages.treasuredata.com/debian' - ) - end - it "/etc/td-agent/td-agent.conf should be in place" do - should contain_file("/etc/td-agent/td-agent.conf").with( - 'ensure' => 'file', - 'owner' => 'root', - 'group' => 'root', - 'notify' => 'Class[Fluentd::Service]' - ).with_content(/^# Include.*config.d.*/m) - end - it "/etc/td-agent/config.d is created" do - should contain_file("/etc/td-agent/config.d").with( - 'ensure' => 'directory', - 'owner' => 'td-agent', - 'group' => 'td-agent', - 'mode' => '0750' - ) - end - it "td-agent package should be installed" do - should contain_package("td-agent").with( - 'ensure' => 'installed' - ) - end - it "td-agent is running" do - should contain_service("td-agent").with( - 'ensure' => 'running', - 'enable' => 'true', - 'hasstatus' => 'true' - ) - end - end - context "Debian without repo installed" do - let (:params) {{ - :package_name => 'td-agent', - :package_ensure => 'installed', - :install_repo => false, - :service_enable => true, - :service_ensure => 'running' - }} - it "has fluentd::packages" do - should contain_class('fluentd::packages') - end - it "has fluentd::config class" do - should contain_class('fluentd::config') - end - it "has fluentd::service class" do - should contain_class('fluentd::service') - end - it "the apt repo file DOES NOT exists" do - should_not contain_apt__source("treasure-data").with( - 'location' => 'http://packages.treasure-data.com/debian' - ) - end - it "/etc/td-agent/td-agent.conf should be in place" do - should contain_file("/etc/td-agent/td-agent.conf").with( - 'ensure' => 'file', - 'owner' => 'root', - 'group' => 'root', - 'notify' => 'Class[Fluentd::Service]' - ).with_content(/^# Include.*config.d.*/m) - end - it "/etc/td-agent/config.d is created" do - should contain_file("/etc/td-agent/config.d").with( - 'ensure' => 'directory', - 'owner' => 'td-agent', - 'group' => 'td-agent', - 'mode' => '0750' - ) - end - it "td-agent package should be installed" do - should contain_package("td-agent").with( - 'ensure' => 'installed' - ) - end - it "td-agent is running" do - should contain_service("td-agent").with( - 'ensure' => 'running', - 'enable' => 'true', - 'hasstatus' => 'true' - ) - end - end - end - context "On a Redhat-like OS with repo installed" do - let (:params) {{ - :package_name => 'td-agent', - :package_ensure => 'installed', - :install_repo => true, - :service_enable => true, - :service_ensure => 'running' - }} - let :facts do - { - :osfamily => 'RedHat', - :operatingsystemrelease => '6.5', - :concat_basedir => '/tmp', - :lsbdistid => 'CentOs', - } - end - - it "has fluentd::packages" do - should contain_class('fluentd::packages') - end - it "has fluentd::config class" do - should contain_class('fluentd::config') - end - it "has fluentd::service class" do - should contain_class('fluentd::service') - end - it "the yum repo file exists" do - should contain_yumrepo('treasuredata') - end - it "/etc/td-agent/td-agent.conf should be in place" do - should contain_file("/etc/td-agent/td-agent.conf").with( - 'ensure' => 'file', - 'owner' => 'root', - 'group' => 'root', - 'notify' => 'Class[Fluentd::Service]' - ).with_content(/^# Include.*config.d.*/m) - end - it "/etc/td-agent/config.d is created" do - should contain_file("/etc/td-agent/config.d").with( - 'ensure' => 'directory', - 'owner' => 'td-agent', - 'group' => 'td-agent', - 'mode' => '0750' - ) - end - it "td-agent package should be installed" do - should contain_package("td-agent").with( - 'ensure' => 'installed' - ) - end - it "td-agent is running" do - should contain_service("td-agent").with( - 'ensure' => 'running', - 'enable' => 'true', - 'hasstatus' => 'true' - ) - end - end - context "On a Redhat-like OS without repo installed" do - let (:params) {{ - :package_name => 'td-agent', - :package_ensure => 'installed', - :install_repo => false, - :service_enable => true, - :service_ensure => 'running' - }} - let :facts do - { - :osfamily => 'RedHat', - :operatingsystemrelease => '6.5', - :concat_basedir => '/tmp', - :lsbdistid => 'CentOs', - } - end - - it "has fluentd::packages" do - should contain_class('fluentd::packages') - end - it "has fluentd::config class" do - should contain_class('fluentd::config') - end - it "has fluentd::service class" do - should contain_class('fluentd::service') - end - it "the yum repo file DOES NOT exists" do - should_not contain_file('/etc/yum.repos.d/td.repo').with_content(/^name=TreasureData$/) - end - it "/etc/td-agent/td-agent.conf should be in place" do - should contain_file("/etc/td-agent/td-agent.conf").with( - 'ensure' => 'file', - 'owner' => 'root', - 'group' => 'root', - 'notify' => 'Class[Fluentd::Service]' - ).with_content(/^# Include.*config.d.*/m) - end - it "/etc/td-agent/config.d is created" do - should contain_file("/etc/td-agent/config.d").with( - 'ensure' => 'directory', - 'owner' => 'td-agent', - 'group' => 'td-agent', - 'mode' => '0750' - ) - end - it "td-agent package should be installed" do - should contain_package("td-agent").with( - 'ensure' => 'installed' - ) - end - it "td-agent is running" do - should contain_service("td-agent").with( - 'ensure' => 'running', - 'enable' => 'true', - 'hasstatus' => 'true' - ) - end - end -end diff --git a/fluentd/spec/defines/fluent_filter_spec.rb b/fluentd/spec/defines/fluent_filter_spec.rb deleted file mode 100644 index 39827cbdf..000000000 --- a/fluentd/spec/defines/fluent_filter_spec.rb +++ /dev/null @@ -1,39 +0,0 @@ -#!/usr/bin/env rspec -require 'spec_helper' - -describe 'fluentd::filter' do - let(:title) {'bar'} - - let (:facts) {{ - :osfamily => 'Debian', - :concat_basedir => '/dne', - :lsbdistid => 'Debian', - }} - - context "when type is grep" do - let(:params) {{ - :configfile => 'foo', - :pattern => 'baz', - :type => 'grep', - :input_key => 'input_key', - :regexp => '/regex/', - :exclude => 'exclude', - :config => {}, - :output_tag => 'output_tag', - :add_tag_prefix => 'add_tag_prefix', - :remove_tag_prefix => 'remove_tag_prefix', - :add_tag_suffix => 'add_tag_suffix', - :remove_tag_suffix => 'remove_tag_suffix', - }} - - it "should install the grep plugin" do - should contain_fluentd__install_plugin('fluent-plugin-grep') - end - - - it "should create matcher single segment" do - should contain_concat__fragment('filter').with_content(/.*type.*input_key.*regexp.*exclude.*output_tag.*add_tag_prefix.*remove_tag_prefix.*add_tag_suffix.*remove_tag_suffix.*<\/match>/m) - end - end - -end \ No newline at end of file diff --git a/fluentd/spec/defines/fluentd_configfile_spec.rb b/fluentd/spec/defines/fluentd_configfile_spec.rb deleted file mode 100644 index 6f3bb2675..000000000 --- a/fluentd/spec/defines/fluentd_configfile_spec.rb +++ /dev/null @@ -1,30 +0,0 @@ -#!/usr/bin/env rspec -require 'spec_helper' - -describe 'fluentd::configfile', :type => :define do - let :pre_condition do - 'include fluentd' - end - ['Debian','RedHat'].each do |osfam| - context "When on an #{osfam} system" do - let (:facts) {{ - :osfamily => osfam, - :concat_basedir => '/dne', - :lsbdistid => 'Debian', # Concatlib needs this value. Works for RedHat too. - }} - context 'when fed no parameters' do - let (:title) { 'MyBaconBringsAllTheBoysToTheYard'} - context 'provides a stub config' do - it { should contain_class('fluentd') } - it { should contain_concat("/etc/td-agent/config.d/#{title}.conf").with( - 'owner' => 'td-agent', - 'group' => 'td-agent', - 'mode' => '0644', - 'require' => 'Class[Fluentd::Packages]' - ) - } - end - end - end - end -end \ No newline at end of file diff --git a/fluentd/spec/defines/fluentd_match_spec.rb b/fluentd/spec/defines/fluentd_match_spec.rb deleted file mode 100644 index 2c459add6..000000000 --- a/fluentd/spec/defines/fluentd_match_spec.rb +++ /dev/null @@ -1,76 +0,0 @@ -#!/usr/bin/env rspec -require 'spec_helper' - -describe 'fluentd::match' do - let(:title) {'bar'} - - let (:facts) {{ - :osfamily => 'Debian', - :concat_basedir => '/dne', - :lsbdistid => 'Debian', - }} - - context "when no servers or out_copy" do - let(:params) {{ - :configfile => 'foo', - :pattern => 'baz', - :type => 'file', - :config => { - 'time_slice_wait' => '10m', - 'compress' => 'gzip', - } - }} - - it "should create matcher single segment" do - should contain_concat__fragment('match_bar').with_content(/.*type.*file.*compress.*gzip.*time_slice_wait.*10m.*<\/match>/m) - should_not contain_concat__fragment('match_bar').with_content(/server/) - should_not contain_concat__fragment('match_bar').with_content(/store/) - end - end - - context "when servers but no out_copy" do - let(:params) {{ - :configfile => 'foo', - :pattern => 'baz', - :type => 'file', - :servers => [{ 'host' => 'kelis', 'port' => '24224'}, { 'host' => 'bossy', 'port' => '24224'}], - :config => { - 'time_slice_wait' => '10m', - 'compress' => 'gzip', - } - }} - - it "should create matcher with server" do - should contain_concat__fragment('match_bar').with_content(/.*type.*file.*compress.*gzip.*time_slice_wait.*10m.*.*host kelis.*port.*24224.*<\/server>.*.*host.*bossy.*port.*24224.*<\/server>.*<\/match>/m) - should contain_concat__fragment('match_bar').with_content(/server/) - should_not contain_concat__fragment('match_bar').with_content(/store/) - end - end - - context "when out_copy" do - let(:params) {{ - :configfile => 'foo', - :pattern => 'baz', - :type => 'copy', - :config => [ - { - 'type' => 'file', - 'compress' => 'gzip', - 'servers' => [{ 'host' => 'kelis', 'port' => '24224'}, { 'host' => 'bossy', 'port' => '24224'}], - }, - { - 'type' => 'mongo', - 'database' => 'dummy', - } - ] - }} - - it "should create matcher with server" do - should contain_concat__fragment('match_bar').with_content(/.*type.*copy.*.*compress.*gzip.*.*host.*kelis.*port.*24224.*<\/server>.*.*host.*bossy.*port.*24224.*<\/server>.*type.*file.*<\/store>.*.*database.*dummy.*type.*mongo.*<\/store>.*<\/match>/m) - end - end - - -end - - diff --git a/fluentd/spec/fixtures/unit/provider/fluentgem/gem-list-single-package b/fluentd/spec/fixtures/unit/provider/fluentgem/gem-list-single-package deleted file mode 100644 index 41576e3e6..000000000 --- a/fluentd/spec/fixtures/unit/provider/fluentgem/gem-list-single-package +++ /dev/null @@ -1,4 +0,0 @@ - -*** REMOTE GEMS *** - -bundler (1.6.2) diff --git a/fluentd/spec/fixtures/unit/provider/fluentgem/line-with-1.8.5-warning b/fluentd/spec/fixtures/unit/provider/fluentgem/line-with-1.8.5-warning deleted file mode 100644 index 07a27e7b8..000000000 --- a/fluentd/spec/fixtures/unit/provider/fluentgem/line-with-1.8.5-warning +++ /dev/null @@ -1,14 +0,0 @@ -/home/jenkins/.rvm/gems/ruby-1.8.5-p231@global/gems/rubygems-bundler-0.9.0/lib/rubygems-bundler/regenerate_binstubs_command.rb:34: warning: parenthesize argument(s) for future version - -*** LOCAL GEMS *** - -columnize (0.3.2) -diff-lcs (1.1.3) -metaclass (0.0.1) -mocha (0.10.5) -rake (0.8.7) -rspec-core (2.9.0) -rspec-expectations (2.9.1) -rspec-mocks (2.9.0) -rubygems-bundler (0.9.0) -rvm (1.11.3.3) diff --git a/fluentd/spec/spec.opts b/fluentd/spec/spec.opts deleted file mode 100644 index de653df4b..000000000 --- a/fluentd/spec/spec.opts +++ /dev/null @@ -1,4 +0,0 @@ ---format s ---colour ---loadby mtime ---backtrace diff --git a/fluentd/spec/spec_helper.rb b/fluentd/spec/spec_helper.rb deleted file mode 100644 index 479ed6922..000000000 --- a/fluentd/spec/spec_helper.rb +++ /dev/null @@ -1,2 +0,0 @@ -require 'rubygems' -require 'puppetlabs_spec_helper/module_spec_helper' \ No newline at end of file diff --git a/fluentd/spec/unit/provider/fluentgem_spec.rb b/fluentd/spec/unit/provider/fluentgem_spec.rb deleted file mode 100644 index 1e324ae94..000000000 --- a/fluentd/spec/unit/provider/fluentgem_spec.rb +++ /dev/null @@ -1,176 +0,0 @@ -#! /usr/bin/env ruby -require 'spec_helper' - -provider_class = Puppet::Type.type(:package).provider(:gem) - -describe provider_class do - let(:resource) do - Puppet::Type.type(:package).new( - :name => 'myresource', - :ensure => :installed - ) - end - - let(:provider) do - provider = provider_class.new - provider.resource = resource - provider - end - - before :each do - resource.provider = provider - end - - describe "when installing" do - it "should use the path to the gem" do - provider_class.stubs(:command).with(:gemcmd).returns "/my/gem" - provider.expects(:execute).with { |args| args[0] == "/my/gem" }.returns "" - provider.install - end - - it "should specify that the gem is being installed" do - provider.expects(:execute).with { |args| args[1] == "install" }.returns "" - provider.install - end - - it "should specify that documentation should not be included" do - provider.expects(:execute).with { |args| args[2] == "--no-rdoc" }.returns "" - provider.install - end - - it "should specify that RI should not be included" do - provider.expects(:execute).with { |args| args[3] == "--no-ri" }.returns "" - provider.install - end - - it "should specify the package name" do - provider.expects(:execute).with { |args| args[4] == "myresource" }.returns "" - provider.install - end - - describe "when a source is specified" do - describe "as a normal file" do - it "should use the file name instead of the gem name" do - resource[:source] = "/my/file" - provider.expects(:execute).with { |args| args[2] == "/my/file" }.returns "" - provider.install - end - end - describe "as a file url" do - it "should use the file name instead of the gem name" do - resource[:source] = "file:///my/file" - provider.expects(:execute).with { |args| args[2] == "/my/file" }.returns "" - provider.install - end - end - describe "as a puppet url" do - it "should fail" do - resource[:source] = "puppet://my/file" - lambda { provider.install }.should raise_error(Puppet::Error) - end - end - describe "as a non-file and non-puppet url" do - it "should treat the source as a gem repository" do - resource[:source] = "http://host/my/file" - provider.expects(:execute).with { |args| args[2..4] == ["--source", "http://host/my/file", "myresource"] }.returns "" - provider.install - end - end - describe "with an invalid uri" do - it "should fail" do - URI.expects(:parse).raises(ArgumentError) - resource[:source] = "http:::::uppet:/:/my/file" - lambda { provider.install }.should raise_error(Puppet::Error) - end - end - end - end - - describe "#latest" do - it "should return a single value for 'latest'" do - #gemlist is used for retrieving both local and remote version numbers, and there are cases - # (particularly local) where it makes sense for it to return an array. That doesn't make - # sense for '#latest', though. - provider.class.expects(:gemlist).with({ :justme => 'myresource'}).returns({ - :name => 'myresource', - :ensure => ["3.0"], - :provider => :gem, - }) - provider.latest.should == "3.0" - end - - it "should list from the specified source repository" do - resource[:source] = "http://foo.bar.baz/gems" - provider.class.expects(:gemlist). - with({:justme => 'myresource', :source => "http://foo.bar.baz/gems"}). - returns({ - :name => 'myresource', - :ensure => ["3.0"], - :provider => :gem, - }) - provider.latest.should == "3.0" - end - end - - describe "#instances" do - before do - provider_class.stubs(:command).with(:gemcmd).returns "/my/gem" - end - - it "should return an empty array when no gems installed" do - provider_class.expects(:execute).with(%w{/my/gem list --local}).returns("\n") - provider_class.instances.should == [] - end - - it "should return ensure values as an array of installed versions" do - provider_class.expects(:execute).with(%w{/my/gem list --local}).returns <<-HEREDOC.gsub(/ /, '') - systemu (1.2.0) - vagrant (0.8.7, 0.6.9) - HEREDOC - - provider_class.instances.map {|p| p.properties}.should == [ - {:ensure => ["1.2.0"], :provider => :gem, :name => 'systemu'}, - {:ensure => ["0.8.7", "0.6.9"], :provider => :gem, :name => 'vagrant'} - ] - end - - it "should ignore platform specifications" do - provider_class.expects(:execute).with(%w{/my/gem list --local}).returns <<-HEREDOC.gsub(/ /, '') - systemu (1.2.0) - nokogiri (1.6.1, 1.4.4.1) - HEREDOC - - provider_class.instances.map {|p| p.properties}.should == [ - {:ensure => ["1.2.0"], :provider => :gem, :name => 'systemu'}, - {:ensure => ["1.6.1", "1.4.4.1"], :provider => :gem, :name => 'nokogiri'} - ] - end - - it "should not fail when an unmatched line is returned" do - provider_class.expects(:execute).with(%w{/my/gem list --local}). - returns(File.read(my_fixture('line-with-1.8.5-warning'))) - - provider_class.instances.map {|p| p.properties}. - should == [{:provider=>:gem, :ensure=>["0.3.2"], :name=>"columnize"}, - {:provider=>:gem, :ensure=>["1.1.3"], :name=>"diff-lcs"}, - {:provider=>:gem, :ensure=>["0.0.1"], :name=>"metaclass"}, - {:provider=>:gem, :ensure=>["0.10.5"], :name=>"mocha"}, - {:provider=>:gem, :ensure=>["0.8.7"], :name=>"rake"}, - {:provider=>:gem, :ensure=>["2.9.0"], :name=>"rspec-core"}, - {:provider=>:gem, :ensure=>["2.9.1"], :name=>"rspec-expectations"}, - {:provider=>:gem, :ensure=>["2.9.0"], :name=>"rspec-mocks"}, - {:provider=>:gem, :ensure=>["0.9.0"], :name=>"rubygems-bundler"}, - {:provider=>:gem, :ensure=>["1.11.3.3"], :name=>"rvm"}] - end - end - - #describe "listing gems" do - # describe "searching for a single package" do - # it "searches for an exact match" do - # provider_class.expects(:execute).with(includes('^bundler$')).returns(File.read(my_fixture('gem-list-single-package'))) - # expected = {:name => 'bundler', :ensure => %w[1.6.2], :provider => :fluentgem} - # expect(provider_class.gemlist({:justme => 'bundler'})).to eq(expected) - # end - # end - #end -end diff --git a/fluentd/spec_helper.rb b/fluentd/spec_helper.rb deleted file mode 100644 index 3d9200524..000000000 --- a/fluentd/spec_helper.rb +++ /dev/null @@ -1 +0,0 @@ -require 'puppetlabs_spec_helper/module_spec_helper' \ No newline at end of file diff --git a/fluentd/templates/filter.erb b/fluentd/templates/filter.erb deleted file mode 100644 index b5b059eef..000000000 --- a/fluentd/templates/filter.erb +++ /dev/null @@ -1,30 +0,0 @@ -> - type <%= @type %> -<%- if @input_key != '' -%> - input_key <%= @input_key %> -<%- if @regexp != '' -%> - regexp <%= @regexp %> -<%- end -%> -<%- if @exclude != '' -%> - exclude <%= @exclude %> -<%- end -%> -<%- end -%> -<%- @config.sort_by{|key,val|key}.each do |key, val| -%> - <%= key %> <%= val %> -<%- end -%> -<%- if @output_tag != '' -%> - tag <%= @output_tag %> -<%- end -%> -<%- if @add_tag_prefix != '' -%> - add_tag_prefix <%= @add_tag_prefix %> -<%- end -%> -<%- if @remove_tag_prefix != '' -%> - remove_tag_prefix <%= @remove_tag_prefix %> -<%- end -%> -<%- if @add_tag_suffix != '' -%> - add_tag_suffix <%= @add_tag_suffix %> -<%- end -%> -<%- if @remove_tag_suffix != '' -%> - remove_tag_suffix <%= @remove_tag_suffix %> -<%- end -%> - diff --git a/fluentd/templates/forest_match.erb b/fluentd/templates/forest_match.erb deleted file mode 100644 index 2b7053b25..000000000 --- a/fluentd/templates/forest_match.erb +++ /dev/null @@ -1,21 +0,0 @@ -> - type <%= @type %> -<% @config.sort_by{|key,val|key}.each do |key, val| -%> - <% if val.class == Hash && key == 'template' %> - - <% else -%> - <%= key %> <%= val %> - <% end -%> -<% end -%> -<% @servers.each do |server| -%> - -<% @server.sort_by{|key,val|key}.each do |key, val| -%> - <%= key %> <%= val %> -<% end -%> - -<% end -%> - \ No newline at end of file diff --git a/fluentd/templates/match.erb b/fluentd/templates/match.erb deleted file mode 100644 index 1fc09f9c5..000000000 --- a/fluentd/templates/match.erb +++ /dev/null @@ -1,33 +0,0 @@ -> - type <%= @type %> - <%- if @type == 'copy' -%> - <%- @config.each do |outcopy| -%> - - <%- outcopy.sort_by{|key,val|key}.each do |key, val| -%> - <%- if key == 'servers' -%> - <%- val.each do |server| -%> - - <%- server.sort_by{|k,v|k}.each do |k,v| -%> - <%= k %> <%= v %> - <%- end -%> - - <%- end -%> - <%- else -%> - <%= key %> <%= val %> - <%- end -%> - <%- end -%> - - <%- end -%> - <%- else -%> - <%- @config.sort_by{|key,val|key}.each do |key, val| -%> - <%= key %> <%= val %> - <%- end -%> - <%- end -%> - <%- @servers.each do |server| -%> - - <%- server.sort_by{|key,val|key}.each do |key, val| -%> - <%= key %> <%= val %> - <%- end -%> - - <%- end -%> - diff --git a/fluentd/templates/source.erb b/fluentd/templates/source.erb deleted file mode 100644 index ea5a2aa88..000000000 --- a/fluentd/templates/source.erb +++ /dev/null @@ -1,16 +0,0 @@ - - type <%= @type %> -<% @config.sort_by{|key,val|key}.each do |key, val| -%> - <%= key %> <%= val %> -<% end -%> -<% @pattern.each do |pat| -%> - -<% pat.sort_by{|key,val|key}.each do |key, val| -%> - <%= key %> <%= val %> -<% end -%> - -<% end -%> -<% if @format != false -%> format <%= @format %><% end %> -<% if @time_format != false -%> time_format <%= @time_format %><% end %> -<% if @tag != false -%> tag <%= @tag %><% end %> - diff --git a/fluentd/templates/td-agent.conf.erb b/fluentd/templates/td-agent.conf.erb deleted file mode 100644 index dfced4343..000000000 --- a/fluentd/templates/td-agent.conf.erb +++ /dev/null @@ -1,2 +0,0 @@ -# Include config files in the ./config.d directory -<% if @version == '2' -%>@<% end %>include config.d/*.conf From 4f2c3ceafe1e782eda6182b152ed1c8b8d7ca81c Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Wed, 16 Dec 2015 08:53:02 -0500 Subject: [PATCH 06/35] Add soylent/konstantin-fluentd module. The upstream module is now: https://github.com/soylent/konstantin-fluentd This module update commit was generated by Bade. For more info please check https://github.com/paramite/bade This commit is setting modules to following state: fluentd - initial commit: b462da7d1c6290afba38fd7e64226990ecf795c7 Change-Id: I2e6af8e44d382cf1e5baa5fabf4b2fc0e542d357 (cherry picked from commit 19fd15cd83bc96e3016f4432fb3ce24f238f263f) --- Puppetfile | 4 + fluentd/.fixtures.yml | 6 + fluentd/.gitignore | 5 + fluentd/CHANGELOG.md | 28 +++ fluentd/Gemfile | 9 + fluentd/LICENSE | 176 +++++++++++++++ fluentd/README.md | 212 ++++++++++++++++++ fluentd/Rakefile | 17 ++ fluentd/examples/init.pp | 19 ++ fluentd/files/td-agent.conf | 2 + .../puppet/parser/functions/fluent_config.rb | 29 +++ .../lib/puppet/provider/package/tdagent.rb | 3 + fluentd/manifests/config.pp | 10 + fluentd/manifests/init.pp | 46 ++++ fluentd/manifests/install.pp | 18 ++ fluentd/manifests/install_repo.pp | 47 ++++ fluentd/manifests/params.pp | 41 ++++ fluentd/manifests/plugin.pp | 12 + fluentd/manifests/service.pp | 10 + fluentd/metadata.json | 47 ++++ fluentd/spec/acceptance/class_spec.rb | 20 ++ .../acceptance/nodesets/debian-78-x64.yml | 11 + fluentd/spec/acceptance/nodesets/default.yml | 11 + .../nodesets/ubuntu-server-1404-x64.yml | 11 + fluentd/spec/classes/init_spec.rb | 24 ++ fluentd/spec/classes/install_spec.rb | 33 +++ fluentd/spec/classes/service_spec.rb | 17 ++ fluentd/spec/defines/config_spec.rb | 66 ++++++ fluentd/spec/defines/plugin_spec.rb | 20 ++ fluentd/spec/spec_helper.rb | 1 + fluentd/spec/spec_helper_acceptance.rb | 19 ++ 31 files changed, 974 insertions(+) create mode 100644 fluentd/.fixtures.yml create mode 100644 fluentd/.gitignore create mode 100644 fluentd/CHANGELOG.md create mode 100644 fluentd/Gemfile create mode 100644 fluentd/LICENSE create mode 100644 fluentd/README.md create mode 100644 fluentd/Rakefile create mode 100644 fluentd/examples/init.pp create mode 100644 fluentd/files/td-agent.conf create mode 100644 fluentd/lib/puppet/parser/functions/fluent_config.rb create mode 100644 fluentd/lib/puppet/provider/package/tdagent.rb create mode 100644 fluentd/manifests/config.pp create mode 100644 fluentd/manifests/init.pp create mode 100644 fluentd/manifests/install.pp create mode 100644 fluentd/manifests/install_repo.pp create mode 100644 fluentd/manifests/params.pp create mode 100644 fluentd/manifests/plugin.pp create mode 100644 fluentd/manifests/service.pp create mode 100644 fluentd/metadata.json create mode 100644 fluentd/spec/acceptance/class_spec.rb create mode 100644 fluentd/spec/acceptance/nodesets/debian-78-x64.yml create mode 100644 fluentd/spec/acceptance/nodesets/default.yml create mode 100644 fluentd/spec/acceptance/nodesets/ubuntu-server-1404-x64.yml create mode 100644 fluentd/spec/classes/init_spec.rb create mode 100644 fluentd/spec/classes/install_spec.rb create mode 100644 fluentd/spec/classes/service_spec.rb create mode 100644 fluentd/spec/defines/config_spec.rb create mode 100644 fluentd/spec/defines/plugin_spec.rb create mode 100644 fluentd/spec/spec_helper.rb create mode 100644 fluentd/spec/spec_helper_acceptance.rb diff --git a/Puppetfile b/Puppetfile index a27b08e9b..a16a68f11 100644 --- a/Puppetfile +++ b/Puppetfile @@ -58,6 +58,10 @@ mod 'firewall', :commit => '1b6cc9192150d9521cc70301d0452daf189a63f7', :git => 'https://github.com/puppetlabs/puppetlabs-firewall.git' +mod 'fluentd', + :commit => 'b462da7d1c6290afba38fd7e64226990ecf795c7', + :git => 'https://github.com/soylent/konstantin-fluentd.git' + mod 'galera', :commit => 'a2ecf273aef9ef9193bba35c235fb7b8b821a0c3', :git => 'https://github.com/redhat-openstack/puppet-galera.git' diff --git a/fluentd/.fixtures.yml b/fluentd/.fixtures.yml new file mode 100644 index 000000000..d73a0c5d5 --- /dev/null +++ b/fluentd/.fixtures.yml @@ -0,0 +1,6 @@ +fixtures: + repositories: + stdlib: "git://github.com/puppetlabs/puppetlabs-stdlib.git" + apt: "git://github.com/puppetlabs/puppetlabs-apt.git" + symlinks: + fluentd: "#{source_dir}" diff --git a/fluentd/.gitignore b/fluentd/.gitignore new file mode 100644 index 000000000..1ac0a277b --- /dev/null +++ b/fluentd/.gitignore @@ -0,0 +1,5 @@ +.vagrant/ +Gemfile.lock +log/ +pkg/ +spec/fixtures/ diff --git a/fluentd/CHANGELOG.md b/fluentd/CHANGELOG.md new file mode 100644 index 000000000..3989a7df5 --- /dev/null +++ b/fluentd/CHANGELOG.md @@ -0,0 +1,28 @@ +## Unreleased + +## 2015-12-02 - Release v. 0.3.1 + + - Add param `repo_desc` + +## 2015-10-28 - Release v. 0.3.0 + + - Remove class `fluentd::config` + - Add defined type `fluentd::config` + - Add defined type `fluentd::plugin` + +## 2015-10-22 - Release v. 0.2.0 + + - Add param `service_manage` + - Add param `repo_gpgkeyid` + - Add param `repo_install` + - Add param `plugin_source` + - Rename param `repo_baseurl` to `repo_url` + - Remove param `config_template` + - Param validation + - Support Ubuntu 14.04 + - Support Debian 7.8 + - Support nested config tags + +## 2015-10-19 - Release v. 0.1.0 + + - Initial release diff --git a/fluentd/Gemfile b/fluentd/Gemfile new file mode 100644 index 000000000..b14d3f3a6 --- /dev/null +++ b/fluentd/Gemfile @@ -0,0 +1,9 @@ +source 'https://rubygems.org' + +gem 'puppet', ENV.fetch('PUPPET_VERSION', '>= 3.3') +gem 'rake' +gem 'puppet-lint', '>= 1.0.0' +gem 'puppetlabs_spec_helper', '>= 0.8.2' +gem 'beaker-rspec' +gem 'beaker-puppet_install_helper' +gem 'metadata-json-lint' diff --git a/fluentd/LICENSE b/fluentd/LICENSE new file mode 100644 index 000000000..d0381d6d0 --- /dev/null +++ b/fluentd/LICENSE @@ -0,0 +1,176 @@ +Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS diff --git a/fluentd/README.md b/fluentd/README.md new file mode 100644 index 000000000..40283d33b --- /dev/null +++ b/fluentd/README.md @@ -0,0 +1,212 @@ +# Fluentd + +Install, configure, and manage Fluentd data collector. + +## Module Description + +* Installs `td-agent` package +* Generates configuration file `td-agent.conf` +* Generates custom configuration files and saves them to `config.d/` +* Manages `td-agent` service +* Installs Fluentd gem plugins + +## Usage + +### Routing events to Elasticsearch + +```puppet +include fluentd + +fluentd::plugin { 'fluent-plugin-elasticsearch': } + +fluentd::config { '500_elasticsearch.conf': + config => { + 'source' => { + 'type' => 'unix', + 'path' => '/tmp/td-agent/td-agent.sock', + }, + 'match' => { + 'tag_pattern' => '**', + 'type' => 'elasticsearch', + 'index_name' => 'foo', + 'type_name' => 'bar', + 'logstash_format' => true, + } + } +} +``` + +### Forwarding events to Fluentd aggregator + +```puppet +include fluentd + +fluentd::config { '600_forwarding.conf': + config => { + 'source' => { + 'type' => unix, + 'path' => '/tmp/td-agent/td-agent.sock', + }, + 'match' => { + 'tag_pattern' => '**', + 'type' => forward, + 'server' => [ + { 'host' => 'example1.com', 'port' => 24224 }, + { 'host' => 'example2.com', 'port' => 24224 }, + ] + } + } +} +``` + +### Config File Naming + +All configs employ a numbering system in the resource's title that is used for +ordering. When titling your config, make sure you prefix the filename with a +number, for example, '999_catch_all.conf', '500_elasticsearch.conf'. 999 has +smaller priority than 500. + +## Reference + +### Classes + +#### Public Classes + +* `fluentd`: Main class, includes all other classes. + +#### Private Classes + +* `fluentd::install`: Handles the packages. +* `fluentd::service`: Handles the service. + +### Parameters + +The following parameters are available in the `fluentd` class: + +#### `repo_install` + +Default value: true + +#### `repo_name` + +Default value: 'treasuredata' + +#### `repo_desc` + +Default value: 'TreasureData' + +#### `repo_url` + +Default value: 'http://packages.treasuredata.com/2/redhat/$releasever/$basearch' + +#### `repo_enabled` + +Default value: true + +#### `repo_gpgcheck` + +Default value: true + +#### `repo_gpgkey` + +Default value: 'https://packages.treasuredata.com/GPG-KEY-td-agent' + +#### `repo_gpgkeyid` + +Default value: 'C901622B5EC4AF820C38AB861093DB45A12E206F' + +#### `package_name` + +Default value: 'td-agent' + +#### `package_ensure` + +Default value: present + +#### `service_name` + +Default value: 'td-agent' + +#### `service_ensure` + +Default value: running + +#### `service_enable` + +Default value: true + +#### `service_manage` + +Default value: true + +#### `config_file` + +Default value: '/etc/td-agent/td-agent.conf' + +### Public Defines + +* `fluentd::config`: Generates custom configuration files. +* `fluentd::plugin`: Installs plugins. + +The following parameters are available in the `fluentd::plugin` defined type: + +#### `title` + +Plugin name + +#### `plugin_ensure` + +Default value: present + +#### `plugin_source` + +Default value: 'https://rubygems.org' + +The following parameters are available in the `fluentd::config` defined type: + +#### `title` + +Config filename + +#### `config` + +Config Hash, please see usage examples. + +## Limitations + +Tested only on CentOS 7, Ubuntu 14.04, Debian 7.8 + +## Development + +Bug reports and pull requests are welcome! + +### Running Tests + + $ bundle install + $ bundle exec rspec + $ bundle exec rake beaker BEAKER_set=debian-78-x64 + $ bundle exec rake beaker BEAKER_set=ubuntu-server-1404-x64 + $ bundle exec rake lint + +Relevant Beaker docs: https://github.com/puppetlabs/beaker/blob/master/docs/How-to-Write-a-Beaker-Test-for-a-Module.md + +### TODO: + +* Remove `rubygems` package dependency + +## License + +Copyright 2015 SPB TV AG + +Licensed under the Apache License, Version 2.0 (the "License"); you may not use +this file except in compliance with the License. + +You may obtain a copy of the License at +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software distributed +under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR +CONDITIONS OF ANY KIND, either express or implied. + +See the License for the specific language governing permissions and limitations +under the License. diff --git a/fluentd/Rakefile b/fluentd/Rakefile new file mode 100644 index 000000000..150a669d9 --- /dev/null +++ b/fluentd/Rakefile @@ -0,0 +1,17 @@ +require 'puppetlabs_spec_helper/rake_tasks' +require 'puppet-lint/tasks/puppet-lint' + +PuppetLint.configuration.ignore_paths = ['spec/**/*.pp', 'pkg/**/*.pp'] + +desc 'Validate manifests, templates, and ruby files' +task :validate do + Dir['manifests/**/*.pp'].each do |manifest| + sh "puppet parser validate --noop #{manifest}" + end + Dir['spec/**/*.rb','lib/**/*.rb'].each do |ruby_file| + sh "ruby -c #{ruby_file}" unless ruby_file =~ /spec\/fixtures/ + end + Dir['templates/**/*.erb'].each do |template| + sh "erb -P -x -T '-' #{template} | ruby -c" + end +end diff --git a/fluentd/examples/init.pp b/fluentd/examples/init.pp new file mode 100644 index 000000000..c21eb9c7d --- /dev/null +++ b/fluentd/examples/init.pp @@ -0,0 +1,19 @@ +include fluentd + +fluentd::plugin { 'fluent-plugin-elasticsearch': } + +fluentd::config { '500_elasticsearch.conf': + config => { + 'source' => { + 'type' => 'unix', + 'path' => '/tmp/td-agent/td-agent.sock', + }, + 'match' => { + 'tag_pattern' => '**', + 'type' => 'elasticsearch', + 'index_name' => 'foo', + 'type_name' => 'bar', + 'logstash_format' => true, + } + } +} diff --git a/fluentd/files/td-agent.conf b/fluentd/files/td-agent.conf new file mode 100644 index 000000000..c66251959 --- /dev/null +++ b/fluentd/files/td-agent.conf @@ -0,0 +1,2 @@ +# td-agent.conf, managed by Puppet. +@include 'config.d/*.conf' diff --git a/fluentd/lib/puppet/parser/functions/fluent_config.rb b/fluentd/lib/puppet/parser/functions/fluent_config.rb new file mode 100644 index 000000000..ac86d86c4 --- /dev/null +++ b/fluentd/lib/puppet/parser/functions/fluent_config.rb @@ -0,0 +1,29 @@ +module Puppet::Parser::Functions + newfunction(:fluent_plugin_config, type: :rvalue) do |args| + plugin_type = args[0] + plugin_config = args[1] + + tag_pattern = plugin_config.delete('tag_pattern') + + config_body = plugin_config.each_with_object('') do |(key, value), result| + if value.is_a?(Array) + value.each do |plugin_sub_config| + result << function_fluent_plugin_config([key, plugin_sub_config]) + end + else + result << [key, value].join(' ') << "\n" + end + end + + "<#{plugin_type} #{tag_pattern}>\n#{config_body}\n\n" + end + + newfunction(:fluent_config, type: :rvalue) do |args| + config = args[0] + + header = "# Managed by Puppet.\n" + config.each_with_object(header) do |(plugin_type, plugin_config), result| + result << function_fluent_plugin_config([plugin_type, plugin_config]) + end.chomp + end +end diff --git a/fluentd/lib/puppet/provider/package/tdagent.rb b/fluentd/lib/puppet/provider/package/tdagent.rb new file mode 100644 index 000000000..ddcdeed52 --- /dev/null +++ b/fluentd/lib/puppet/provider/package/tdagent.rb @@ -0,0 +1,3 @@ +Puppet::Type.type(:package).provide(:tdagent, parent: :gem, source: :gem) do + commands gemcmd: '/opt/td-agent/usr/sbin/td-agent-gem' +end diff --git a/fluentd/manifests/config.pp b/fluentd/manifests/config.pp new file mode 100644 index 000000000..ce7d386ff --- /dev/null +++ b/fluentd/manifests/config.pp @@ -0,0 +1,10 @@ +define fluentd::config($config) { + $path = sprintf('/etc/td-agent/config.d/%s', $title) + + file { $path: + ensure => present, + content => fluent_config($config), + require => Class['Fluentd::Install'], + notify => Class['Fluentd::Service'], + } +} diff --git a/fluentd/manifests/init.pp b/fluentd/manifests/init.pp new file mode 100644 index 000000000..ca8ee064d --- /dev/null +++ b/fluentd/manifests/init.pp @@ -0,0 +1,46 @@ +class fluentd ( + $repo_install = $fluentd::repo_install, + $repo_name = $fluentd::repo_name, + $repo_desc = $fluentd::repo_desc, + $repo_url = $fluentd::repo_url, + $repo_enabled = $fluentd::repo_enabled, + $repo_gpgcheck = $fluentd::repo_gpgcheck, + $repo_gpgkey = $fluentd::repo_gpgkey, + $repo_gpgkeyid = $fluentd::repo_gpgkeyid, + $package_name = $fluentd::package_name, + $package_ensure = $fluentd::package_ensure, + $plugin_names = $fluentd::plugin_names, + $plugin_ensure = $fluentd::plugin_ensure, + $plugin_source = $fluentd::plugin_source, + $service_name = $fluentd::service_name, + $service_ensure = $fluentd::service_ensure, + $service_enable = $fluentd::service_enable, + $service_manage = $fluentd::service_manage, + $config_file = $fluentd::config_file, +) inherits fluentd::params { + + # Param validations + validate_bool($repo_install) + validate_string($repo_name) + validate_string($repo_url) + validate_bool($repo_enabled) + validate_bool($repo_gpgcheck) + validate_string($repo_gpgkey) + validate_string($repo_gpgkeyid) + validate_string($package_name) + validate_string($package_ensure) + validate_array($plugin_names) + validate_string($plugin_ensure) + validate_string($plugin_source) + validate_string($service_name) + validate_string($service_ensure) + validate_bool($service_enable) + validate_bool($service_manage) + validate_absolute_path($config_file) + + contain fluentd::install + contain fluentd::service + + Class['Fluentd::Install'] -> + Class['Fluentd::Service'] +} diff --git a/fluentd/manifests/install.pp b/fluentd/manifests/install.pp new file mode 100644 index 000000000..e5e859774 --- /dev/null +++ b/fluentd/manifests/install.pp @@ -0,0 +1,18 @@ +class fluentd::install inherits fluentd { + if $fluentd::repo_install { + require fluentd::install_repo + } + + package { $fluentd::package_name: + ensure => $fluentd::package_ensure, + } -> + + file { $fluentd::config_path: + ensure => directory, + } -> + + file { $fluentd::config_file: + ensure => present, + content => file('fluentd/td-agent.conf'), + } +} diff --git a/fluentd/manifests/install_repo.pp b/fluentd/manifests/install_repo.pp new file mode 100644 index 000000000..40309641e --- /dev/null +++ b/fluentd/manifests/install_repo.pp @@ -0,0 +1,47 @@ +class fluentd::install_repo inherits fluentd { + case $::osfamily { + 'redhat': { + yumrepo { $fluentd::repo_name: + descr => $fluentd::repo_desc, + baseurl => $fluentd::repo_url, + enabled => $fluentd::repo_enabled, + gpgcheck => $fluentd::repo_gpgcheck, + gpgkey => $fluentd::repo_gpgkey, + notify => Exec['rpmkey'], + } + + exec { 'rpmkey': + command => "rpm --import ${fluentd::repo_gpgkey}", + path => '/usr/bin', + refreshonly => true, + } + + # TODO: Remove this dependency. Gem provider requires this package. + package { 'rubygems': + ensure => present, + } + } + + 'debian': { + apt::source { $fluentd::repo_name: + location => $fluentd::repo_url, + comment => $fluentd::repo_desc, + repos => 'contrib', + architecture => 'amd64', + release => $fluentd::distro_codename, + key => { + id => $fluentd::repo_gpgkeyid, + source => $fluentd::repo_gpgkey, + }, + include => { + 'src' => false, + 'deb' => true, + }, + } + } + + default: { + fail("Unsupported osfamily ${::osfamily}") + } + } +} diff --git a/fluentd/manifests/params.pp b/fluentd/manifests/params.pp new file mode 100644 index 000000000..70572bbb8 --- /dev/null +++ b/fluentd/manifests/params.pp @@ -0,0 +1,41 @@ +class fluentd::params { + $repo_install = true + $repo_name = 'treasuredata' + $repo_desc = 'TreasureData' + + case $::osfamily { + 'redhat': { + $repo_url = 'http://packages.treasuredata.com/2/redhat/$releasever/$basearch' + } + + 'debian': { + $distro_id = downcase($::lsbdistid) + $distro_codename = $::lsbdistcodename + $repo_url = "http://packages.treasuredata.com/2/${distro_id}/${distro_codename}/" + } + + default: { + fail("Unsupported osfamily ${::osfamily}") + } + } + + $repo_enabled = true + $repo_gpgcheck = true + $repo_gpgkey = 'https://packages.treasuredata.com/GPG-KEY-td-agent' + $repo_gpgkeyid = 'C901622B5EC4AF820C38AB861093DB45A12E206F' + + $package_name = 'td-agent' + $package_ensure = present + + $plugin_names = [] + $plugin_ensure = present + $plugin_source = 'https://rubygems.org' + + $service_name = 'td-agent' + $service_ensure = running + $service_enable = true + $service_manage = true + + $config_file = '/etc/td-agent/td-agent.conf' + $config_path = '/etc/td-agent/config.d' +} diff --git a/fluentd/manifests/plugin.pp b/fluentd/manifests/plugin.pp new file mode 100644 index 000000000..f591fbd73 --- /dev/null +++ b/fluentd/manifests/plugin.pp @@ -0,0 +1,12 @@ +define fluentd::plugin( + $plugin_ensure = $fluentd::plugin_ensure, + $plugin_source = $fluentd::plugin_source, +) { + package { $title: + ensure => $plugin_ensure, + source => $plugin_source, + provider => tdagent, + notify => Class['Fluentd::Service'], + require => Class['Fluentd::Install'], + } +} diff --git a/fluentd/manifests/service.pp b/fluentd/manifests/service.pp new file mode 100644 index 000000000..6eafeb584 --- /dev/null +++ b/fluentd/manifests/service.pp @@ -0,0 +1,10 @@ +class fluentd::service inherits fluentd { + if $fluentd::service_manage { + service { $fluentd::service_name: + ensure => $fluentd::service_ensure, + enable => $fluentd::service_enable, + hasstatus => true, + hasrestart => true, + } + } +} diff --git a/fluentd/metadata.json b/fluentd/metadata.json new file mode 100644 index 000000000..91ca80dc5 --- /dev/null +++ b/fluentd/metadata.json @@ -0,0 +1,47 @@ +{ + "author": "konstantin", + "dependencies": [ + { + "name": "puppetlabs/stdlib", + "version_requirement": ">= 1.0.0" + }, + { + "name": "puppetlabs/apt", + "version_requirement": ">= 1.0.0" + } + ], + "issues_url": "https://github.com/soylent/konstantin-fluentd/issues", + "license": "Apache-2.0", + "name": "konstantin-fluentd", + "operatingsystem_support": [ + { + "operatingsystem": "CentOS", + "operatingsystemrelease": [ + "7.0" + ] + }, + { + "operatingsystem": "Ubuntu", + "operatingsystemrelease": [ + "14.04" + ] + }, + { + "operatingsystem": "Debian", + "operatingsystemrelease": [ + "7" + ] + } + ], + "project_page": "https://github.com/soylent/konstantin-fluentd", + "source": "https://github.com/soylent/konstantin-fluentd", + "summary": "Installs, configures, and manages the Fluentd service", + "tags": [ + "fluentd", + "fluent", + "td-agent", + "treasuredata", + "logging" + ], + "version": "0.3.1" +} diff --git a/fluentd/spec/acceptance/class_spec.rb b/fluentd/spec/acceptance/class_spec.rb new file mode 100644 index 000000000..d9d5694ce --- /dev/null +++ b/fluentd/spec/acceptance/class_spec.rb @@ -0,0 +1,20 @@ +require 'spec_helper_acceptance' + +RSpec.describe 'fluentd' do + it 'runs successfully' do + manifest = File.read(File.expand_path('../../examples/init.pp', __dir__)) + + # Run it twice and test for idempotency + apply_manifest(manifest, catch_failures: true) + expect(apply_manifest(manifest, catch_failures: true).exit_code).to be_zero + end + + describe package('td-agent') do + it { is_expected.to be_installed } + end + + describe service('td-agent') do + it { is_expected.to be_enabled.with_level(3) } + it { is_expected.to be_running } + end +end diff --git a/fluentd/spec/acceptance/nodesets/debian-78-x64.yml b/fluentd/spec/acceptance/nodesets/debian-78-x64.yml new file mode 100644 index 000000000..762164d67 --- /dev/null +++ b/fluentd/spec/acceptance/nodesets/debian-78-x64.yml @@ -0,0 +1,11 @@ +HOSTS: + debian-78-x64: + roles: + - master + platform: debian-7-amd64 + box: puppetlabs/debian-7.8-64-nocm + box_url: https://vagrantcloud.com/puppetlabs/boxes/debian-7.8-64-nocm + hypervisor: vagrant +CONFIG: + log_level: verbose + type: foss diff --git a/fluentd/spec/acceptance/nodesets/default.yml b/fluentd/spec/acceptance/nodesets/default.yml new file mode 100644 index 000000000..ec395054d --- /dev/null +++ b/fluentd/spec/acceptance/nodesets/default.yml @@ -0,0 +1,11 @@ +HOSTS: + centos-70-x64: + roles: + - master + platform: el-7-x86_64 + box: puppetlabs/centos-7.0-64-nocm + box_url: https://atlas.hashicorp.com/puppetlabs/boxes/centos-7.0-64-nocm + hypervisor: vagrant +CONFIG: + log_level: verbose + type: foss diff --git a/fluentd/spec/acceptance/nodesets/ubuntu-server-1404-x64.yml b/fluentd/spec/acceptance/nodesets/ubuntu-server-1404-x64.yml new file mode 100644 index 000000000..7e389e71e --- /dev/null +++ b/fluentd/spec/acceptance/nodesets/ubuntu-server-1404-x64.yml @@ -0,0 +1,11 @@ +HOSTS: + ubuntu-server-1404-x64: + roles: + - master + platform: ubuntu-14.04-amd64 + box: puppetlabs/ubuntu-14.04-64-nocm + box_url: https://vagrantcloud.com/puppetlabs/boxes/ubuntu-14.04-64-nocm + hypervisor: vagrant +CONFIG: + log_level: verbose + type: foss diff --git a/fluentd/spec/classes/init_spec.rb b/fluentd/spec/classes/init_spec.rb new file mode 100644 index 000000000..22a2cc64b --- /dev/null +++ b/fluentd/spec/classes/init_spec.rb @@ -0,0 +1,24 @@ +require 'spec_helper' + +RSpec.describe 'fluentd' do + shared_examples 'works' do + it { is_expected.to compile.with_all_deps } + it { is_expected.to contain_class('fluentd') } + it { is_expected.to contain_class('fluentd::install') } + it { is_expected.to contain_class('fluentd::service') } + end + + context 'when osfamily is debian' do + let(:facts) do + { osfamily: 'Debian', lsbdistid: 'Ubuntu', lsbdistcodename: 'trusty' } + end + + include_examples 'works' + end + + context 'with defaults for all parameters' do + let(:facts) { { osfamily: 'RedHat' } } + + include_examples 'works' + end +end diff --git a/fluentd/spec/classes/install_spec.rb b/fluentd/spec/classes/install_spec.rb new file mode 100644 index 000000000..a5e91563d --- /dev/null +++ b/fluentd/spec/classes/install_spec.rb @@ -0,0 +1,33 @@ +require 'spec_helper' + +RSpec.describe 'fluentd::install' do + shared_examples 'package and configs' do + it { is_expected.to contain_package('td-agent') } + it { is_expected.to contain_file('/etc/td-agent/td-agent.conf') } + it { is_expected.to contain_file('/etc/td-agent/config.d') } + end + + context 'on RedHat based system' do + let(:facts) { { osfamily: 'RedHat' } } + + it { is_expected.to contain_yumrepo('treasuredata') } + + include_examples 'package and configs' + end + + context 'on Debian based system' do + let(:facts) do + { osfamily: 'Debian', lsbdistid: 'Ubuntu', lsbdistcodename: 'trusty' } + end + + it { is_expected.to contain_apt__source('treasuredata') } + + include_examples 'package and configs' + end + + context 'on unsupported system' do + let(:facts) { { osfamily: 'Darwin' } } + + it { is_expected.not_to compile } + end +end diff --git a/fluentd/spec/classes/service_spec.rb b/fluentd/spec/classes/service_spec.rb new file mode 100644 index 000000000..47d3fe69b --- /dev/null +++ b/fluentd/spec/classes/service_spec.rb @@ -0,0 +1,17 @@ +require 'spec_helper' + +RSpec.describe 'fluentd::service' do + context 'on RedHat based system' do + let(:facts) { { osfamily: 'RedHat' } } + + it { is_expected.to contain_service('td-agent') } + end + + context 'on Debian based system' do + let(:facts) do + { osfamily: 'Debian', lsbdistid: 'Ubuntu', lsbdistcodename: 'trusty' } + end + + it { is_expected.to contain_service('td-agent') } + end +end diff --git a/fluentd/spec/defines/config_spec.rb b/fluentd/spec/defines/config_spec.rb new file mode 100644 index 000000000..2d2f86cb6 --- /dev/null +++ b/fluentd/spec/defines/config_spec.rb @@ -0,0 +1,66 @@ +require 'spec_helper' + +RSpec.describe 'fluentd::config' do + let(:pre_condition) { 'include fluentd' } + + context 'on RedHat based system' do + let(:facts) { { osfamily: 'RedHat' } } + + let(:title) { 'stdout.conf' } + + context 'when config does not contain nested hashes' do + let(:params) do + { + config: { + 'source' => { + 'type' => 'forward' + }, + 'match' => { + 'tag_pattern' => '**', + 'type' => 'stdout' + } + } + } + end + + it do + is_expected.to contain_file('/etc/td-agent/config.d/stdout.conf'). + with_content(//). + with_content(/type forward/). + with_content(/<\/source>/). + with_content(//). + with_content(/type stdout/). + with_content(/<\/match>/) + end + end + + context 'when config contains nested hashes' do + let(:params) do + { + config: { + 'match' => { + 'tag_pattern' => '**', + 'type' => 'forward', + 'server' => [ + { 'host' => 'example1.com', 'port' => 24224 }, + { 'host' => 'example2.com', 'port' => 24224 } + ] + } + } + } + end + + it do + is_expected.to contain_file('/etc/td-agent/config.d/stdout.conf'). + with_content(//). + with_content(/type forward/). + with_content(//). + with_content(/host example1.com/). + with_content(/port 24224/). + with_content(/<\/server>/). + with_content(/host example2.com/). + with_content(/<\/match>/) + end + end + end +end diff --git a/fluentd/spec/defines/plugin_spec.rb b/fluentd/spec/defines/plugin_spec.rb new file mode 100644 index 000000000..8483d6e81 --- /dev/null +++ b/fluentd/spec/defines/plugin_spec.rb @@ -0,0 +1,20 @@ +require 'spec_helper' + +RSpec.describe 'fluentd::plugin' do + let(:pre_condition) { 'include fluentd' } + let(:title) { 'fluent-plugin-test' } + + context 'on RedHat based system' do + let(:facts) { { osfamily: 'RedHat' } } + + it { is_expected.to contain_package(title).with(provider: 'tdagent') } + end + + context 'on Debian based system' do + let(:facts) do + { osfamily: 'Debian', lsbdistid: 'Ubuntu', lsbdistcodename: 'trusty' } + end + + it { is_expected.to contain_package(title).with(provider: 'tdagent') } + end +end diff --git a/fluentd/spec/spec_helper.rb b/fluentd/spec/spec_helper.rb new file mode 100644 index 000000000..2c6f56649 --- /dev/null +++ b/fluentd/spec/spec_helper.rb @@ -0,0 +1 @@ +require 'puppetlabs_spec_helper/module_spec_helper' diff --git a/fluentd/spec/spec_helper_acceptance.rb b/fluentd/spec/spec_helper_acceptance.rb new file mode 100644 index 000000000..24570f7ad --- /dev/null +++ b/fluentd/spec/spec_helper_acceptance.rb @@ -0,0 +1,19 @@ +require 'beaker-rspec' +require 'beaker/puppet_install_helper' + +run_puppet_install_helper unless ENV['BEAKER_provision'] == 'no' + +RSpec.configure do |c| + module_root = File.expand_path(File.join(File.dirname(__FILE__), '..')) + + c.formatter = :documentation + + c.before :suite do + puppet_module_install(source: module_root, module_name: 'fluentd') + + hosts.each do |host| + on host, puppet('module', 'install', 'puppetlabs-stdlib'), acceptable_exit_codes: [0, 1] + on host, puppet('module', 'install', 'puppetlabs-apt'), acceptable_exit_codes: [0, 1] + end + end +end From c5941447a544f44750ba02cb09434b563311a8f7 Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Mon, 21 Dec 2015 10:35:51 -0500 Subject: [PATCH 07/35] Update pacemaker to e9a8f0d048c1bbc86a4bb1ee81417da3afe6673e Change-Id: I52a0f97e3877e24206e0c8a64ff3aa375779cb6a --- Puppetfile | 2 +- pacemaker/manifests/corosync.pp | 12 ++++++++++++ pacemaker/manifests/resource/ip.pp | 5 ++++- pacemaker/manifests/stonith/fence_xvm.pp | 21 +++++++++++++++++++-- pacemaker/manifests/stonith/xvm.pp | 19 ++++++++++++++++++- 5 files changed, 54 insertions(+), 5 deletions(-) diff --git a/Puppetfile b/Puppetfile index ca42224bb..30ee32aa0 100644 --- a/Puppetfile +++ b/Puppetfile @@ -187,7 +187,7 @@ mod 'openstacklib', :git => 'https://github.com/openstack/puppet-openstacklib.git' mod 'pacemaker', - :commit => 'e23cbb084bf48c414c189f45a3761c57ba8375d1', + :commit => 'e9a8f0d048c1bbc86a4bb1ee81417da3afe6673e', :git => 'https://github.com/redhat-openstack/puppet-pacemaker.git' mod 'puppet', diff --git a/pacemaker/manifests/corosync.pp b/pacemaker/manifests/corosync.pp index a0fd1e603..50fad1cd1 100644 --- a/pacemaker/manifests/corosync.pp +++ b/pacemaker/manifests/corosync.pp @@ -47,6 +47,12 @@ dport => ['5404', '5405'], action => 'accept', } + firewall { '001 corosync mcast ipv6': + proto => 'udp', + dport => ['5404', '5405'], + action => 'accept', + provider => 'ip6tables', + } } if $pcsd_mode { @@ -56,6 +62,12 @@ dport => ['2224'], action => 'accept', } + firewall { '001 pcsd ipv6': + proto => 'tcp', + dport => ['2224'], + action => 'accept', + provider => 'ip6tables', + } } Service['pcsd'] -> # we have more fragile when-to-start pacemaker conditions with pcsd diff --git a/pacemaker/manifests/resource/ip.pp b/pacemaker/manifests/resource/ip.pp index 09ec0344c..09a0f7ed5 100644 --- a/pacemaker/manifests/resource/ip.pp +++ b/pacemaker/manifests/resource/ip.pp @@ -44,7 +44,10 @@ default => " nic=${nic}" } - pcmk_resource { "ip-${ip_address}": + # pcs dislikes colons from IPv6 addresses. Replacing them with dots. + $resource_name = regsubst($ip_address, '(:)', '.', 'G') + + pcmk_resource { "ip-${resource_name}": ensure => $ensure, resource_type => 'IPaddr2', resource_params => "ip=${ip_address}${cidr_option}${nic_option}", diff --git a/pacemaker/manifests/stonith/fence_xvm.pp b/pacemaker/manifests/stonith/fence_xvm.pp index cba974c7b..e67b108b2 100644 --- a/pacemaker/manifests/stonith/fence_xvm.pp +++ b/pacemaker/manifests/stonith/fence_xvm.pp @@ -103,15 +103,32 @@ proto => 'igmp', action => 'accept', } + firewall { "003 fence_xvm ipv6": + proto => 'igmp', + action => 'accept', + provider => 'ip6tables', + } firewall { "004 fence_xvm": proto => 'udp', - dport => '1229', + dport => '1229', action => 'accept', } + firewall { "004 fence_xvm ipv6": + proto => 'udp', + dport => '1229', + action => 'accept', + provider => 'ip6tables', + } firewall { "005 fence_xvm": proto => 'tcp', - dport => '1229', + dport => '1229', + action => 'accept', + } + firewall { "005 fence_xvm ipv6": + proto => 'tcp', + dport => '1229', action => 'accept', + provider => 'ip6tables', } } diff --git a/pacemaker/manifests/stonith/xvm.pp b/pacemaker/manifests/stonith/xvm.pp index 5aff60ab6..21aa0cfd0 100644 --- a/pacemaker/manifests/stonith/xvm.pp +++ b/pacemaker/manifests/stonith/xvm.pp @@ -36,15 +36,32 @@ proto => 'igmp', action => 'accept', } + firewall { "003 fence_xvm ipv6": + proto => 'igmp', + action => 'accept', + provider => 'ip6tables', + } firewall { "004 fence_xvm": proto => 'udp', - dport => '1229', + dport => '1229', + action => 'accept', + } + firewall { "004 fence_xvm ipv6": + proto => 'udp', + dport => '1229', action => 'accept', + provider => 'ip6tables', } firewall { "005 fence_xvm": + proto => 'tcp', + dport => '1229', + action => 'accept', + } + firewall { "005 fence_xvm ipv6": proto => 'tcp', dport => '1229', action => 'accept', + provider => 'ip6tables', } } From a436889bcfbe8de7dc58ca1fc62f85286c0f3a8d Mon Sep 17 00:00:00 2001 From: Gael Chamoulaud Date: Tue, 22 Dec 2015 12:02:58 +0100 Subject: [PATCH 08/35] Puppet-neutron automatic update This module update commit was generated by Bade. For more info please check https://github.com/paramite/bade This commit is setting modules to following state: neutron - old commit: 0f2cb57696705d3f9dc6f7b4c94f9b469b673486 - new commit: 2b427bb1f9d0b79c49a536ab6046f0095cd6995c Change-Id: I03a5c1bb45dacceab8b4b53ff3904d9941ecc8f0 Resolves: rhbz#1285060 Signed-off-by: Gael Chamoulaud --- Puppetfile | 2 +- neutron/manifests/plugins/nuage.pp | 29 ++++++++++--------- .../classes/neutron_plugins_nuage_spec.rb | 9 ++++++ 3 files changed, 25 insertions(+), 15 deletions(-) diff --git a/Puppetfile b/Puppetfile index ca42224bb..109334ab9 100644 --- a/Puppetfile +++ b/Puppetfile @@ -159,7 +159,7 @@ mod 'nagios', :git => 'https://github.com/gildub/puppet-nagios-openstack.git' mod 'neutron', - :commit => '0f2cb57696705d3f9dc6f7b4c94f9b469b673486', + :commit => '2b427bb1f9d0b79c49a536ab6046f0095cd6995c', :git => 'https://github.com/openstack/puppet-neutron.git' mod 'nova', diff --git a/neutron/manifests/plugins/nuage.pp b/neutron/manifests/plugins/nuage.pp index 7fae1636c..9ed703a1a 100644 --- a/neutron/manifests/plugins/nuage.pp +++ b/neutron/manifests/plugins/nuage.pp @@ -49,19 +49,14 @@ include ::neutron::params + File['/etc/neutron/plugins/nuage/plugin.ini'] -> Neutron_plugin_nuage<||> Neutron_plugin_nuage<||> ~> Service['neutron-server'] + Neutron_plugin_nuage<||> ~> Exec<| title == 'neutron-db-sync' |> file { '/etc/neutron/plugins/nuage': - ensure => 'directory', + ensure => directory, } - ensure_resource('file', $::neutron::params::nuage_config_file, { - ensure => file, - owner => 'root', - group => 'neutron', - mode => '0640'} - ) - if $::osfamily == 'Debian' { file_line { '/etc/default/neutron-server:NEUTRON_PLUGIN_CONFIG': path => '/etc/default/neutron-server', @@ -72,12 +67,22 @@ } if $::osfamily == 'Redhat' { + File['/etc/neutron/plugin.ini'] ~> Exec<| title == 'neutron-db-sync' |> file { '/etc/neutron/plugin.ini': - ensure => link, - target => $::neutron::params::nuage_config_file, + ensure => link, + require => File['/etc/neutron/plugins/nuage/plugin.ini'], + target => $::neutron::params::nuage_config_file, } } + file { '/etc/neutron/plugins/nuage/plugin.ini': + ensure => file, + owner => 'root', + group => 'neutron', + require => File['/etc/neutron/plugins/nuage'], + mode => '0640' + } + $nuage_base_uri_base = '/nuage/api' neutron_plugin_nuage { 'RESTPROXY/default_net_partition_name': value => $nuage_net_partition_name; @@ -94,8 +99,4 @@ ($::neutron::core_plugin != 'nuage') { fail('Nuage plugin should be the core_plugin in neutron.conf') } - - Neutron_plugin_nuage<||> ~> Exec<| title == 'neutron-db-sync' |> - } - diff --git a/neutron/spec/classes/neutron_plugins_nuage_spec.rb b/neutron/spec/classes/neutron_plugins_nuage_spec.rb index 45114718c..636b249f4 100644 --- a/neutron/spec/classes/neutron_plugins_nuage_spec.rb +++ b/neutron/spec/classes/neutron_plugins_nuage_spec.rb @@ -30,6 +30,15 @@ class { 'neutron::server': auth_password => 'password' }" it { is_expected.to contain_class('neutron::params') } + it 'should have a nuage plugin ini file' do + is_expected.to contain_file('/etc/neutron/plugins/nuage/plugin.ini').with( + :ensure => 'file', + :owner => 'root', + :group => 'neutron', + :mode => '0640' + ) + end + it 'should configure neutron.conf' do is_expected.to contain_neutron_config('DEFAULT/core_plugin').with_value('neutron.plugins.nuage.plugin.NuagePlugin') end From 69a5e0033db441b4652c584804d1812b24d659e3 Mon Sep 17 00:00:00 2001 From: Gael Chamoulaud Date: Tue, 22 Dec 2015 14:20:46 +0100 Subject: [PATCH 09/35] Puppet-ironic automatic update This module update commit was generated by Bade. For more info please check https://github.com/paramite/bade This commit is setting modules to following state: ironic - old commit: 0e7546bd7810e10ed3d108a95e6d7673ad3a98ed - new commit: ec08404c8c23f499b972e6348f0bf6c503dc7f3e Resolves: rhbz#1292504 Change-Id: I432fe5de4b8fe14d713950168c1721bc8431b3d6 Signed-off-by: Gael Chamoulaud --- Puppetfile | 2 +- ironic/manifests/drivers/pxe.pp | 42 +++++++++++-------- .../spec/classes/ironic_drivers_pxe_spec.rb | 22 ++++++---- 3 files changed, 38 insertions(+), 28 deletions(-) diff --git a/Puppetfile b/Puppetfile index 30ee32aa0..02a036aad 100644 --- a/Puppetfile +++ b/Puppetfile @@ -99,7 +99,7 @@ mod 'ipa', :git => 'https://github.com/xbezdick/puppet-ipa-1.git' mod 'ironic', - :commit => '0e7546bd7810e10ed3d108a95e6d7673ad3a98ed', + :commit => 'ec08404c8c23f499b972e6348f0bf6c503dc7f3e', :git => 'https://github.com/openstack/puppet-ironic.git' mod 'java', diff --git a/ironic/manifests/drivers/pxe.pp b/ironic/manifests/drivers/pxe.pp index 2df4fb79e..c0cd4b787 100644 --- a/ironic/manifests/drivers/pxe.pp +++ b/ironic/manifests/drivers/pxe.pp @@ -69,30 +69,36 @@ # Should be an valid directory # Defaults to '/var/lib/ironic/master_images'. # +# [*uefi_pxe_bootfile_name*] +# (optional) Bootfile DHCP parameter for UEFI boot mode. +# Defaults to 'elilo.efi'. +# class ironic::drivers::pxe ( - $deploy_kernel = undef, - $deploy_ramdisk = undef, - $pxe_append_params = 'nofb nomodeset vga=normal', - $pxe_config_template = '$pybasedir/drivers/modules/pxe_config.template', - $pxe_deploy_timeout = '0', - $tftp_server = '$my_ip', - $tftp_root = '/tftpboot', - $images_path = '/var/lib/ironic/images/', - $tftp_master_path = '/tftpboot/master_images', - $instance_master_path = '/var/lib/ironic/master_images', + $deploy_kernel = undef, + $deploy_ramdisk = undef, + $pxe_append_params = 'nofb nomodeset vga=normal', + $pxe_config_template = '$pybasedir/drivers/modules/pxe_config.template', + $pxe_deploy_timeout = '0', + $tftp_server = '$my_ip', + $tftp_root = '/tftpboot', + $images_path = '/var/lib/ironic/images/', + $tftp_master_path = '/tftpboot/master_images', + $instance_master_path = '/var/lib/ironic/master_images', + $uefi_pxe_bootfile_name = 'elilo.efi', ) { # Configure ironic.conf ironic_config { - 'pxe/pxe_append_params': value => $pxe_append_params; - 'pxe/pxe_config_template': value => $pxe_config_template; - 'pxe/pxe_deploy_timeout': value => $pxe_deploy_timeout; - 'pxe/tftp_server': value => $tftp_server; - 'pxe/tftp_root': value => $tftp_root; - 'pxe/images_path': value => $images_path; - 'pxe/tftp_master_path': value => $tftp_master_path; - 'pxe/instance_master_path': value => $instance_master_path; + 'pxe/pxe_append_params': value => $pxe_append_params; + 'pxe/pxe_config_template': value => $pxe_config_template; + 'pxe/pxe_deploy_timeout': value => $pxe_deploy_timeout; + 'pxe/tftp_server': value => $tftp_server; + 'pxe/tftp_root': value => $tftp_root; + 'pxe/images_path': value => $images_path; + 'pxe/tftp_master_path': value => $tftp_master_path; + 'pxe/instance_master_path': value => $instance_master_path; + 'pxe/uefi_pxe_bootfile_name': value => $uefi_pxe_bootfile_name; } if $deploy_kernel { diff --git a/ironic/spec/classes/ironic_drivers_pxe_spec.rb b/ironic/spec/classes/ironic_drivers_pxe_spec.rb index 26c13a2c9..64d485ed5 100644 --- a/ironic/spec/classes/ironic_drivers_pxe_spec.rb +++ b/ironic/spec/classes/ironic_drivers_pxe_spec.rb @@ -23,14 +23,15 @@ describe 'ironic::drivers::pxe' do let :default_params do - { :pxe_append_params => 'nofb nomodeset vga=normal', - :pxe_config_template => '$pybasedir/drivers/modules/pxe_config.template', - :pxe_deploy_timeout => '0', - :tftp_server => '$my_ip', - :tftp_root => '/tftpboot', - :images_path => '/var/lib/ironic/images/', - :tftp_master_path => '/tftpboot/master_images', - :instance_master_path => '/var/lib/ironic/master_images' } + { :pxe_append_params => 'nofb nomodeset vga=normal', + :pxe_config_template => '$pybasedir/drivers/modules/pxe_config.template', + :pxe_deploy_timeout => '0', + :tftp_server => '$my_ip', + :tftp_root => '/tftpboot', + :images_path => '/var/lib/ironic/images/', + :tftp_master_path => '/tftpboot/master_images', + :instance_master_path => '/var/lib/ironic/master_images', + :uefi_pxe_bootfile_name => 'elilo.efi' } end let :params do @@ -51,6 +52,7 @@ is_expected.to contain_ironic_config('pxe/images_path').with_value(p[:images_path]) is_expected.to contain_ironic_config('pxe/tftp_master_path').with_value(p[:tftp_master_path]) is_expected.to contain_ironic_config('pxe/instance_master_path').with_value(p[:instance_master_path]) + is_expected.to contain_ironic_config('pxe/uefi_pxe_bootfile_name').with_value(p[:uefi_pxe_bootfile_name]) end context 'when overriding parameters' do @@ -65,7 +67,8 @@ :tftp_root => '/mnt/ftp', :images_path => '/mnt/images', :tftp_master_path => '/mnt/master_images', - :instance_master_path => '/mnt/ironic/master_images' + :instance_master_path => '/mnt/ironic/master_images', + :uefi_pxe_bootfile_name => 'bootx64.efi' ) end @@ -80,6 +83,7 @@ is_expected.to contain_ironic_config('pxe/images_path').with_value(p[:images_path]) is_expected.to contain_ironic_config('pxe/tftp_master_path').with_value(p[:tftp_master_path]) is_expected.to contain_ironic_config('pxe/instance_master_path').with_value(p[:instance_master_path]) + is_expected.to contain_ironic_config('pxe/uefi_pxe_bootfile_name').with_value(p[:uefi_pxe_bootfile_name]) end end From 729c15f390b87ebaa85ee3a858c0a74b0680b8f6 Mon Sep 17 00:00:00 2001 From: Gael Chamoulaud Date: Tue, 22 Dec 2015 15:23:49 +0100 Subject: [PATCH 10/35] Puppet-trove automatic update This module update commit was generated by Bade. For more info please check https://github.com/paramite/bade This commit is setting modules to following state: trove - old commit: 02295f18915107203ef79ef37fe5cc284506cfc1 - new commit: ded110e944204aad0cad7480e445cc241a138f14 Change-Id: Ib8e914105503edbbdc4835b8dfe167c7714921d6 Signed-off-by: Gael Chamoulaud --- Puppetfile | 2 +- trove/templates/trove-guestagent.conf.erb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Puppetfile b/Puppetfile index 9ed70b625..1924ab2cc 100644 --- a/Puppetfile +++ b/Puppetfile @@ -263,7 +263,7 @@ mod 'tripleo', :git => 'https://github.com/openstack/puppet-tripleo.git' mod 'trove', - :commit => '02295f18915107203ef79ef37fe5cc284506cfc1', + :commit => 'ded110e944204aad0cad7480e445cc241a138f14', :git => 'https://github.com/openstack/puppet-trove' mod 'tuskar', diff --git a/trove/templates/trove-guestagent.conf.erb b/trove/templates/trove-guestagent.conf.erb index 5f388709a..bc27479ce 100644 --- a/trove/templates/trove-guestagent.conf.erb +++ b/trove/templates/trove-guestagent.conf.erb @@ -58,7 +58,7 @@ nova_proxy_admin_user = <%= @nova_proxy_admin_user %> nova_proxy_admin_pass = <%= @nova_proxy_admin_pass %> nova_proxy_admin_tenant_name = <%= @nova_proxy_admin_tenant_name %> trove_auth_url = <%= @auth_url %> -swift_url = http://localhost:8080/v1/AUTH_ +swift_url = <%= @swift_url %> # Datastore management implementations. Format datastore:manager.impl # datastore_registry_ext = mysql:trove.guestagent.datastore.mysql.manager.Manager, percona:trove.guestagent.datastore.mysql.manager.Manager From 423383ee83b85b7ea8b0a4f3e4dd6d49f1302433 Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Wed, 6 Jan 2016 15:35:20 -0500 Subject: [PATCH 11/35] Add puppetlabs-git module. This is needed as a dependency for some of the monitoring toolchain modules. This module update commit was generated by Bade. For more info please check https://github.com/paramite/bade This commit is setting modules to following state: git - initial commit: 8e7f586 Change-Id: Ieccdb1aca8af53c4f94354a889e6211921c5058d --- Puppetfile | 4 + git/.fixtures.yml | 5 + git/.gitignore | 7 + git/.travis.yml | 21 ++ git/CHANGELOG.md | 56 +++++ git/Gemfile | 53 +++++ git/LICENSE | 201 ++++++++++++++++++ git/README.md | 107 ++++++++++ git/Rakefile | 1 + git/files/subtree/bash_completion.sh | 25 +++ git/lib/facter/git_exec_path.rb | 25 +++ git/lib/facter/git_html_path.rb | 24 +++ git/lib/facter/git_version.rb | 21 ++ .../parser/functions/git_config_hash.rb | 25 +++ .../puppet/provider/git_config/git_config.rb | 53 +++++ git/lib/puppet/type/git_config.rb | 67 ++++++ git/manifests/config.pp | 73 +++++++ git/manifests/gitosis.pp | 13 ++ git/manifests/init.pp | 43 ++++ git/manifests/subtree.pp | 57 +++++ git/metadata.json | 56 +++++ git/spec/acceptance/class_spec.rb | 21 ++ .../git_config_different_users_spec.rb | 44 ++++ .../acceptance/git_config_provider_spec.rb | 29 +++ .../acceptance/nodesets/centos-64-x64.yml | 11 + git/spec/acceptance/nodesets/default.yml | 12 ++ .../nodesets/ubuntu-server-12042-x64.yml | 12 ++ git/spec/classes/git_gitosis_spec.rb | 10 + git/spec/classes/git_spec.rb | 70 ++++++ git/spec/classes/git_subtree_spec.rb | 77 +++++++ git/spec/defines/git_config_spec.rb | 37 ++++ git/spec/spec_helper.rb | 1 + git/spec/spec_helper_acceptance.rb | 24 +++ git/spec/unit/facter/git_exec_path_spec.rb | 27 +++ git/spec/unit/facter/git_html_path_spec.rb | 27 +++ git/spec/unit/facter/git_version_spec.rb | 35 +++ .../provider/git_config/git_config_spec.rb | 14 ++ .../puppet/type/git_config/git_config_spec.rb | 49 +++++ git/tests/gitosis.pp | 1 + git/tests/init.pp | 1 + 40 files changed, 1439 insertions(+) create mode 100644 git/.fixtures.yml create mode 100644 git/.gitignore create mode 100644 git/.travis.yml create mode 100644 git/CHANGELOG.md create mode 100644 git/Gemfile create mode 100644 git/LICENSE create mode 100644 git/README.md create mode 100644 git/Rakefile create mode 100644 git/files/subtree/bash_completion.sh create mode 100644 git/lib/facter/git_exec_path.rb create mode 100644 git/lib/facter/git_html_path.rb create mode 100644 git/lib/facter/git_version.rb create mode 100644 git/lib/puppet/parser/functions/git_config_hash.rb create mode 100644 git/lib/puppet/provider/git_config/git_config.rb create mode 100644 git/lib/puppet/type/git_config.rb create mode 100644 git/manifests/config.pp create mode 100644 git/manifests/gitosis.pp create mode 100644 git/manifests/init.pp create mode 100644 git/manifests/subtree.pp create mode 100644 git/metadata.json create mode 100644 git/spec/acceptance/class_spec.rb create mode 100644 git/spec/acceptance/git_config_different_users_spec.rb create mode 100644 git/spec/acceptance/git_config_provider_spec.rb create mode 100644 git/spec/acceptance/nodesets/centos-64-x64.yml create mode 100644 git/spec/acceptance/nodesets/default.yml create mode 100644 git/spec/acceptance/nodesets/ubuntu-server-12042-x64.yml create mode 100644 git/spec/classes/git_gitosis_spec.rb create mode 100644 git/spec/classes/git_spec.rb create mode 100644 git/spec/classes/git_subtree_spec.rb create mode 100644 git/spec/defines/git_config_spec.rb create mode 100644 git/spec/spec_helper.rb create mode 100644 git/spec/spec_helper_acceptance.rb create mode 100644 git/spec/unit/facter/git_exec_path_spec.rb create mode 100644 git/spec/unit/facter/git_html_path_spec.rb create mode 100644 git/spec/unit/facter/git_version_spec.rb create mode 100644 git/spec/unit/puppet/provider/git_config/git_config_spec.rb create mode 100644 git/spec/unit/puppet/type/git_config/git_config_spec.rb create mode 100644 git/tests/gitosis.pp create mode 100644 git/tests/init.pp diff --git a/Puppetfile b/Puppetfile index 1924ab2cc..f227a6ce0 100644 --- a/Puppetfile +++ b/Puppetfile @@ -66,6 +66,10 @@ mod 'galera', :commit => 'a2ecf273aef9ef9193bba35c235fb7b8b821a0c3', :git => 'https://github.com/redhat-openstack/puppet-galera.git' +mod 'git', + :commit => '8e7f586', + :git => 'https://github.com/puppetlabs/puppetlabs-git.git' + mod 'glance', :commit => 'f4495d93a064ea8d1f5a023c156ab73e14e69198', :git => 'https://github.com/openstack/puppet-glance.git' diff --git a/git/.fixtures.yml b/git/.fixtures.yml new file mode 100644 index 000000000..2aa9e3d77 --- /dev/null +++ b/git/.fixtures.yml @@ -0,0 +1,5 @@ +fixtures: + repositories: + vcsrepo: git://github.com/puppetlabs/puppetlabs-vcsrepo.git + symlinks: + git: "#{source_dir}" diff --git a/git/.gitignore b/git/.gitignore new file mode 100644 index 000000000..34894fa6b --- /dev/null +++ b/git/.gitignore @@ -0,0 +1,7 @@ +pkg/ +Gemfile.lock +vendor/ +spec/fixtures/ +.vagrant/ +.bundle/ +coverage/ \ No newline at end of file diff --git a/git/.travis.yml b/git/.travis.yml new file mode 100644 index 000000000..c0d90693f --- /dev/null +++ b/git/.travis.yml @@ -0,0 +1,21 @@ +--- +language: ruby +bundler_args: --without development system_tests +script: "bundle exec rake validate && bundle exec rake lint && bundle exec rake spec SPEC_OPTS='--format documentation'" +matrix: + fast_finish: true + include: + - rvm: 1.8.7 + env: PUPPET_GEM_VERSION="~> 3.0" FACTER_GEM_VERSION="~> 1.0" + - rvm: 1.8.7 + env: PUPPET_GEM_VERSION="~> 3.0" FACTER_GEM_VERSION="~> 2.0" + - rvm: 1.9.3 + env: PUPPET_GEM_VERSION="~> 3.0" FACTER_GEM_VERSION="~> 1.0" + - rvm: 1.9.3 + env: PUPPET_GEM_VERSION="~> 3.0" FACTER_GEM_VERSION="~> 2.0" + - rvm: 2.0.0 + env: PUPPET_GEM_VERSION="~> 3.0" FACTER_GEM_VERSION="~> 1.0" + - rvm: 2.0.0 + env: PUPPET_GEM_VERSION="~> 3.0" FACTER_GEM_VERSION="~> 2.0" +notifications: + email: false diff --git a/git/CHANGELOG.md b/git/CHANGELOG.md new file mode 100644 index 000000000..004c37d00 --- /dev/null +++ b/git/CHANGELOG.md @@ -0,0 +1,56 @@ +##2015-05-26 - Release 0.4.0 +###Summary +This release adds greater flexibility to `git` and `git_config` and includes a couple of bug fixes, including fixing `git_config` with multiple users. + +####Deprecations +- The `section` parameter in `git_config` and `git::config` has been deprecated. The full option name should be passed to the `key` parameter instead (i.e., "user.email") + +####Features +- Refactored existing facts and added spec tests (MODULES-1571) +- Test and doc updates +- New parameters in class `git`: + - `package_manage` + - `package_ensure` + - `configs` + +####Bugfixes +- Only run if git is actually installed (MODULES-1238) +- Fix `git_config` to work with multiple users (MODULES-1863) + +##2014-11-18 - Release 0.3.0 +###Summary +This release primarily includes improvements to `git::config` and the addition of the `git_config` type&provider, along with much improved testing + +####Features +- Add `user` and `scope` parameter to `git::config` +- Add `git_config` type +- Refactor `git::config` to use `git_config` +- Test improvements + +####Bugfixes +- Redirect stderr to the correct place on windows + +##2014-07-15 - Release 0.2.0 +###Summary +This release updates metadata.json so the module can be uninstalled and +upgraded via the puppet module command. It also lets you set the +`package_name`. + +####Features +- Ability to set `package_name` + +##2014-06-25 - Release 0.1.0 +###Summary +This release adds git::subtree and git::config, as well as fixes up the +documentation and unit tests. + +####Features +- README improvements. +- Add git::subtree class to install git-subtree. +- Add git::config resource + +####Bugfixes +- Fix git_version fact. + +##2011-06-03 - Release 0.0.1 +- Initial commit diff --git a/git/Gemfile b/git/Gemfile new file mode 100644 index 000000000..b9665d6b1 --- /dev/null +++ b/git/Gemfile @@ -0,0 +1,53 @@ +source ENV['GEM_SOURCE'] || "https://rubygems.org" + +def location_for(place, fake_version = nil) + if place =~ /^(git:[^#]*)#(.*)/ + [fake_version, { :git => $1, :branch => $2, :require => false }].compact + elsif place =~ /^file:\/\/(.*)/ + ['>= 0', { :path => File.expand_path($1), :require => false }] + else + [place, { :require => false }] + end +end + +is_ruby18 = RUBY_VERSION.start_with? '1.8' + +group :development, :test do + gem 'rake', :require => false + if is_ruby18 + gem 'rspec', "~> 3.1.0", :require => false + end + gem 'rspec-puppet', :require => false + gem 'puppetlabs_spec_helper', :require => false + gem 'puppet-lint', :require => false + gem 'pry', :require => false + gem 'simplecov', :require => false + gem 'metadata-json-lint', :require => false +end + +beaker_version = ENV['BEAKER_VERSION'] +group :system_tests do + gem 'serverspec', :require => false + if beaker_version + gem 'beaker', *location_for(beaker_version) + else + gem 'beaker', :require => false + end + gem 'beaker-rspec', :require => false +end + +facter_version = ENV['FACTER_GEM_VERSION'] +if facter_version + gem 'facter', *location_for(facter_version) +else + gem 'facter', :require => false +end + +puppet_version = ENV['PUPPET_GEM_VERSION'] +if puppet_version + gem 'puppet', *location_for(puppet_version) +else + gem 'puppet', :require => false +end + +# vim:ft=ruby diff --git a/git/LICENSE b/git/LICENSE new file mode 100644 index 000000000..297f85cfa --- /dev/null +++ b/git/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright 2013 Puppet Labs + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/git/README.md b/git/README.md new file mode 100644 index 000000000..32f5c1dcf --- /dev/null +++ b/git/README.md @@ -0,0 +1,107 @@ +#git + +####Table of Contents + +1. [Overview - What is the git module?](#overview) +2. [Module Description - What does the module do?](#module-description) +3. [Setup - The basics of getting started with git](#setup) + * [What git affects](#what-registry-affects) +4. [Usage - Configuration options and additional functionality](#usage) +6. [Limitations - OS compatibility, etc.](#limitations) +7. [Development - Guide for contributing to the module](#development) + +##Overview + +Simple module that can install git or gitosis + +##Module Description + +This module installs the git revision control system on a target node. It does not manage a git server or any associated services; it simply ensures a bare minimum set of features (e.g. just a package) to use git. + +##Setup + +###What git affects + +* Package['git'] + +The specifics managed by the module may vary depending on the platform. + +##Usage + +###I just want `git` installed +Simply include the `git` class. + +```puppet +include git +``` + +###I want to configure `git` using hiera + +```yaml +git::configs: + user.name: 'test' + user.email: 'test@example.com' + core.filemode: + value: false + scope: system +``` + +If using the flat config syntax, options common to all items can be set: + +```yaml +git::configs: + core.filemode: false +git::configs_defaults: + scope: system +``` + +###I want to use `git subtree` with bash completion + +```puppet +include git::subtree +``` + +###I want to set my user.name and user.email + +```puppet +git::config { 'user.name': + value => 'John Doe', +} + +git::config { 'user.email': + value => 'john.doe@example.com', +} +``` + +##Reference + +###Classes + +* `git`: Installs the git client package. +* `gitosis`: Installs the gitosis package. No configuration +* `subtree`: Installs and configures git-subtree for git 1.7 and up. + +###Resources + +* `git::config`: Set git global configuration for the user running puppet, for the specified `$user` or for the system. + +###Facts + +* `git_exec_path`: Path to the directory containing all `git-*` commands. +* `git_version`: Version of git that is installed. Undefined if not installed. + +##Limitations + +This module is known to work with the following operating system families: + + - RedHat 5, 6 + - Debian 6.0.7 or newer + - Ubuntu 12.04 or newer + +##Development + +Puppet Labs modules on the Puppet Forge are open projects, and community contributions are essential for keeping them great. We can’t access the huge number of platforms and myriad of hardware, software, and deployment configurations that Puppet is intended to serve. + +We want to keep it as easy as possible to contribute changes so that our modules work in your environment. There are a few guidelines that we need contributors to follow so that we can have a chance of keeping on top of things. + +You can read the complete module contribution guide [on the Puppet Labs wiki.](http://projects.puppetlabs.com/projects/module-site/wiki/Module_contributing) diff --git a/git/Rakefile b/git/Rakefile new file mode 100644 index 000000000..cd3d37995 --- /dev/null +++ b/git/Rakefile @@ -0,0 +1 @@ +require 'puppetlabs_spec_helper/rake_tasks' diff --git a/git/files/subtree/bash_completion.sh b/git/files/subtree/bash_completion.sh new file mode 100644 index 000000000..f2683e449 --- /dev/null +++ b/git/files/subtree/bash_completion.sh @@ -0,0 +1,25 @@ +#!bash +# +# bash completion support for Git subtree. +# +# To use this routine: +# +# 1) Make sure you have installed and configured the core Git completion script, which is required to make this script work; +# 2) Copy this file to somewhere (e.g. ~/.git-subtree-completion.sh); +# 3) Added the following line to your .bashrc: +# source ~/.git-subtree-completion.sh +# + +_git_subtree () +{ + local cur="${COMP_WORDS[COMP_CWORD]}" + + if [ $COMP_CWORD -eq 2 ]; then + __gitcomp "add merge pull push split" + return + elif [ $COMP_CWORD -eq 3 ]; then + __gitcomp "--prefix=" + return + fi + __gitcomp "$(__git_remotes)" +} diff --git a/git/lib/facter/git_exec_path.rb b/git/lib/facter/git_exec_path.rb new file mode 100644 index 000000000..3a4ad8f04 --- /dev/null +++ b/git/lib/facter/git_exec_path.rb @@ -0,0 +1,25 @@ +# Fact: git_exec_path +# +# Purpose: get git's exec path +# +# Resolution: +# Uses git's --exec-path flag +# +# Caveats: +# none +# +# Notes: +# None +Facter.add('git_exec_path') do + case Facter.value(:osfamily) + when 'windows' + null_path = 'nul' + else + null_path = '/dev/null' + end + git_exec_path_cmd = "git --exec-path 2>#{null_path}" + setcode do + Facter::Util::Resolution.exec(git_exec_path_cmd) + end +end + diff --git a/git/lib/facter/git_html_path.rb b/git/lib/facter/git_html_path.rb new file mode 100644 index 000000000..471fed730 --- /dev/null +++ b/git/lib/facter/git_html_path.rb @@ -0,0 +1,24 @@ +# Fact: git_html_path +# +# Purpose: get git's html path +# +# Resolution: +# Uses git's --html-path flag +# +# Caveats: +# none +# +# Notes: +# None +Facter.add('git_html_path') do + case Facter.value(:osfamily) + when 'windows' + null_path = 'nul' + else + null_path = '/dev/null' + end + git_html_path_cmd = "git --html-path 2>#{null_path}" + setcode do + Facter::Util::Resolution.exec(git_html_path_cmd) + end +end diff --git a/git/lib/facter/git_version.rb b/git/lib/facter/git_version.rb new file mode 100644 index 000000000..c0ece6ae9 --- /dev/null +++ b/git/lib/facter/git_version.rb @@ -0,0 +1,21 @@ +# Fact: git_version +# +# Purpose: get git's current version +# +# Resolution: +# Uses git's --version flag and parses the result from 'version' +# +# Caveats: +# none +# +# Notes: +# None +Facter.add('git_version') do + setcode do + if Facter::Util::Resolution.which('git') + git_version_cmd = 'git --version 2>&1' + git_version_result = Facter::Util::Resolution.exec(git_version_cmd) + git_version_result.to_s.lines.first.strip.split(/version/)[1].strip + end + end +end diff --git a/git/lib/puppet/parser/functions/git_config_hash.rb b/git/lib/puppet/parser/functions/git_config_hash.rb new file mode 100644 index 000000000..edb6fddac --- /dev/null +++ b/git/lib/puppet/parser/functions/git_config_hash.rb @@ -0,0 +1,25 @@ +# +# config_hash.rb +# + +module Puppet::Parser::Functions + newfunction(:git_config_hash, :type => :rvalue, :doc => <<-EOS +This function ensures the proper structure for git configuration options. +*Examples:* + git_config_hash({"foo" => 1, "bar" => {"value" => 2}}) +Would return: {"foo" => {"value" => 1}, "bar" => {"value" => 2}} + EOS + ) do |arguments| + + raise(Puppet::ParseError, "git_config_hash(): Wrong number of arguments " + + "given (#{arguments.size} for 1)") if arguments.size < 1 + + configs = arguments[0] + + unless configs.is_a?(Hash) + raise(Puppet::ParseError, 'git_config_hash(): Requires hash to work with') + end + + return Hash[configs.map {|k, v| [k, v.is_a?(Hash) ? v : {"value" => v}] }] + end +end diff --git a/git/lib/puppet/provider/git_config/git_config.rb b/git/lib/puppet/provider/git_config/git_config.rb new file mode 100644 index 000000000..c0e9d6331 --- /dev/null +++ b/git/lib/puppet/provider/git_config/git_config.rb @@ -0,0 +1,53 @@ +require "shellwords" + +Puppet::Type.type(:git_config).provide(:git_config) do + + mk_resource_methods + + def value + require 'etc' + user = @property_hash[:user] = @resource[:user] + key = @property_hash[:key] = @resource[:key] + section = @property_hash[:section] = @resource[:section] + scope = @property_hash[:scope] = @resource[:scope] + home = Etc.getpwnam(user)[:dir] + + # Backwards compatibility with deprecated $section parameter. + if section && !section.empty? + key = "#{section}.#{key}" + end + + current = Puppet::Util::Execution.execute( + "cd / ; git config --#{scope} --get #{key}", + :uid => user, + :failonfail => false, + :combine => true, + :custom_environment => { 'HOME' => home } + ) + @property_hash[:value] = current.strip + @property_hash[:value] + end + + def value=(value) + require 'etc' + user = @resource[:user] + key = @resource[:key] + section = @resource[:section] + scope = @resource[:scope] + home = Etc.getpwnam(user)[:dir] + + # Backwards compatibility with deprecated $section parameter. + if section && !section.empty? + key = "#{section}.#{key}" + end + + Puppet::Util::Execution.execute( + "cd / ; git config --#{scope} #{key} #{value.shellescape}", + :uid => user, + :failonfail => true, + :combine => true, + :custom_environment => { 'HOME' => home } + ) + end + +end diff --git a/git/lib/puppet/type/git_config.rb b/git/lib/puppet/type/git_config.rb new file mode 100644 index 000000000..55c330e52 --- /dev/null +++ b/git/lib/puppet/type/git_config.rb @@ -0,0 +1,67 @@ +Puppet::Type.newtype(:git_config) do + + desc <<-DOC + Used to configure git + === Examples + + + git_config { 'user.name': + value => 'John Doe', + } + + git_config { 'user.email': + value => 'john.doe@example.com', + } + + git_config { 'user.name': + value => 'Mike Color', + user => 'vagrant', + require => Class['git'], + } + + git_config { 'http.sslCAInfo': + value => $companyCAroot, + user => 'root', + scope => 'system', + require => Company::Certificate['companyCAroot'], + } + DOC + + validate do + fail('it is required to pass "value"') if self[:value].nil? || self[:value].empty? || self[:value] == :absent + warning('Parameter `section` is deprecated, supply the full option name (e.g. "user.email") in the `key` parameter') if + self[:section] && !self[:section].empty? + end + + newparam(:name, :namevar => true) do + desc "The name of the config" + end + + newproperty(:value) do + desc "The config value. Example Mike Color or john.doe@example.com" + end + + newparam(:user) do + desc "The user for which the config will be set. Default value: root" + defaultto "root" + end + + newparam(:key) do + desc "The configuration key. Example: user.email." + end + + autorequire(:user) do + self[:user] + end + + newparam(:section) do + desc "Deprecated: the configuration section. For example, to set user.email, use section => \"user\", key => \"email\"." + defaultto "" + end + + newparam(:scope) do + desc "The scope of the configuration, can be system or global. Default value: global" + defaultto "global" + end + +end diff --git a/git/manifests/config.pp b/git/manifests/config.pp new file mode 100644 index 000000000..a582f489f --- /dev/null +++ b/git/manifests/config.pp @@ -0,0 +1,73 @@ +# == Define: git::config +# +# Used to configure git +# +# === Parameters +# +# [*value*] +# The config value. Example: Mike Color or john.doe@example.com. +# See examples below. +# +# [*key*] +# The name of the option to be set. Example: user.email. +# Default value: same as resource name. +# +# [*user*] +# The user for which the config will be set. Default value: root +# +# [*scope*] +# The scope of the configuration, can be system or global. +# Default value: global +# +# === Examples +# +# Provide some examples on how to use this type: +# +# git::config { 'user.name': +# value => 'John Doe', +# } +# +# git::config { 'user.email': +# value => 'john.doe@example.com', +# } +# +# git::config { 'user.name': +# value => 'Mike Color', +# user => 'vagrant', +# require => Class['git'], +# } +# +# git::config { 'http.sslCAInfo': +# value => $companyCAroot, +# user => 'root', +# scope => 'system', +# require => Company::Certificate['companyCAroot'], +# } +# +# === Authors +# +# === Copyright +# +define git::config ( + $value, + $key = $name, + $section = undef, + $user = 'root', + $scope = 'global', +) { + # Backwards compatibility with deprecated $section parameter. + # (Old versions took separate $section and $key, e.g. "user" and "email".) + if $section != undef { + warning('Parameter `section` is deprecated, supply the full option name (e.g. "user.email") in the `key` parameter') + $_key = "${section}.${key}" + } else { + $_key = $key + } + + git_config { $title: + key => $_key, + value => $value, + user => $user, + scope => $scope, + } +} diff --git a/git/manifests/gitosis.pp b/git/manifests/gitosis.pp new file mode 100644 index 000000000..64b7b2df3 --- /dev/null +++ b/git/manifests/gitosis.pp @@ -0,0 +1,13 @@ +# Class: gitosis +# +# This installs and configures gitosis +# +# Requires: +# - Class[git] +# +class git::gitosis { + include ::git + package {'gitosis': + ensure => present + } +} diff --git a/git/manifests/init.pp b/git/manifests/init.pp new file mode 100644 index 000000000..77a79610c --- /dev/null +++ b/git/manifests/init.pp @@ -0,0 +1,43 @@ +# Class: git +# +# This class installs and configures git +# +# Actions: +# - Install the git package +# - Configure git +# +# Sample Usage: +# class { 'git': } +# +# === Parameters +# +# [*package_ensure*] +# Value to be passed to ensure in the package resource. Defaults to installed. +# +# [*package_manage*] +# boolean toggle to overide the management of the git package. +# You may want to change this behavior if another module manages git packages +# defaults to true +# +# [*configs*] +# hash of configurations as per the git::config defined type +# +# [*configs_defaults*] +# hash of configuration defaults as per the git::config defined type +# to use for every *configs* item +# +class git ( + $package_name = 'git', + $package_ensure = 'installed', + $package_manage = true, + $configs = {}, + $configs_defaults = {} +) { + if ( $package_manage ) { + package { $package_name: + ensure => $package_ensure, + } + } + + create_resources(git::config, git_config_hash($configs), $configs_defaults) +} diff --git a/git/manifests/subtree.pp b/git/manifests/subtree.pp new file mode 100644 index 000000000..199b6dfe0 --- /dev/null +++ b/git/manifests/subtree.pp @@ -0,0 +1,57 @@ +# == Class: git::subtree +# +# Installs and configure git-subtree +# +class git::subtree { + + include ::git + + Package['git'] -> Class['git::subtree'] + + if (versioncmp('1.7.0', $::git_version) > 0) { + fail 'git-subtree requires git 1.7 or later!' + } + + if (versioncmp('1.7.11', $::git_version) > 0) { + $source_dir = '/usr/src/git-subtree' + vcsrepo { $source_dir: + ensure => present, + source => 'https://github.com/apenwarr/git-subtree.git', + provider => 'git', + revision => '2793ee6ba', + before => Exec['Build git-subtree'], + } + } else { + $source_dir = "${::git_html_path}/contrib/subtree" + } + + exec { 'Build git-subtree': + command => "make prefix=/usr libexecdir=${::git_exec_path}", + creates => "${source_dir}/git-subtree", + cwd => $source_dir, + path => ['/usr/bin', '/bin', '/usr/local/bin'], + } + -> + package { [ 'asciidoc', 'xmlto', ]: + ensure => present, + } + -> + exec { 'Install git-subtree': + command => "make prefix=/usr libexecdir=${::git_exec_path} install", + onlyif => [ + "test ! -f ${::git_exec_path}/git-subtree", + 'test ! -f /usr/share/man/man1/git-subtree.1', + ], + cwd => $source_dir, + path => ['/usr/bin', '/bin', '/usr/local/bin'], + } + + file { '/etc/bash_completion.d/git-subtree': + ensure => file, + source => 'puppet:///modules/git/subtree/bash_completion.sh', + owner => 'root', + group => 'root', + mode => '0644', + } + +} diff --git a/git/metadata.json b/git/metadata.json new file mode 100644 index 000000000..a8a76c68a --- /dev/null +++ b/git/metadata.json @@ -0,0 +1,56 @@ +{ + "name": "puppetlabs-git", + "version": "0.4.0", + "author": "Puppet Labs", + "summary": "Module for installing Git or Gitosis.", + "license": "Apache-2.0", + "source": "https://github.com/puppetlabs/puppetlabs-git", + "project_page": "https://github.com/puppetlabs/puppetlabs-git", + "issues_url": "https://tickets.puppetlabs.com/browse/MODULES", + "operatingsystem_support": [ + { + "operatingsystem": "RedHat", + "operatingsystemrelease": [ + "5", + "6", + "7" + ] + }, + { + "operatingsystem": "CentOS", + "operatingsystemrelease": [ + "5", + "6", + "7" + ] + }, + { + "operatingsystem": "Debian", + "operatingsystemrelease": [ + "6", + "7" + ] + }, + { + "operatingsystem": "Ubuntu", + "operatingsystemrelease": [ + "10.04", + "12.04", + "14.04" + ] + } + ], + "requirements": [ + { + "name": "pe", + "version_requirement": "3.x" + }, + { + "name": "puppet", + "version_requirement": "3.x" + } + ], + "dependencies": [ + {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0"} + ] +} diff --git a/git/spec/acceptance/class_spec.rb b/git/spec/acceptance/class_spec.rb new file mode 100644 index 000000000..1f8471081 --- /dev/null +++ b/git/spec/acceptance/class_spec.rb @@ -0,0 +1,21 @@ +require 'spec_helper_acceptance' + +describe 'git class' do + + context 'default parameters' do + # Using puppet_apply as a helper + it 'should work idempotently with no errors' do + pp = <<-EOS + class { 'git': } + EOS + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + apply_manifest(pp, :catch_changes => true) + end + + describe package('git') do + it { should be_installed } + end + end +end diff --git a/git/spec/acceptance/git_config_different_users_spec.rb b/git/spec/acceptance/git_config_different_users_spec.rb new file mode 100644 index 000000000..4c4d5e142 --- /dev/null +++ b/git/spec/acceptance/git_config_different_users_spec.rb @@ -0,0 +1,44 @@ +require 'spec_helper_acceptance' + +describe 'git::config class' do + + context 'with some user settings' do + it 'should work idempotently with no errors' do + pp = <<-EOS + package { 'git': } + -> + git::config { 'Root User Email': + key => 'user.email', + value => 'john.doe@example.com', + } + -> + user { 'janedoe': + ensure => 'present', + managehome => true, + } + -> + file { '/home/janedoe/.gitconfig': + ensure => 'present', + } + -> + git::config { 'Jane User Email': + key => 'user.email', + user => 'janedoe', + value => 'jane.doe@example.com', + } + EOS + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + apply_manifest(pp, :catch_changes => true) + end + + describe file('/home/janedoe/.gitconfig') do + its(:content) { should match /email = jane.doe@example.com/ } + end + + describe file('/root/.gitconfig') do + its(:content) { should match /email = john.doe@example.com/ } + end + end +end diff --git a/git/spec/acceptance/git_config_provider_spec.rb b/git/spec/acceptance/git_config_provider_spec.rb new file mode 100644 index 000000000..ca9863d30 --- /dev/null +++ b/git/spec/acceptance/git_config_provider_spec.rb @@ -0,0 +1,29 @@ +require 'spec_helper_acceptance' + +describe 'git::config class' do + + context 'with some user settings' do + it 'should work idempotently with no errors' do + pp = <<-EOS + package { 'git': } + -> + git::config { 'user.name': + value => 'John Doe', + } + -> + git::config { 'user.email': + value => 'john.doe@example.com', + } + EOS + + # Run it twice and test for idempotency + apply_manifest(pp, :catch_failures => true) + apply_manifest(pp, :catch_changes => true) + end + + describe file('/root/.gitconfig') do + its(:content) { should match /email = john.doe@example.com/ } + its(:content) { should match /name = John Doe/ } + end + end +end diff --git a/git/spec/acceptance/nodesets/centos-64-x64.yml b/git/spec/acceptance/nodesets/centos-64-x64.yml new file mode 100644 index 000000000..d19aa6951 --- /dev/null +++ b/git/spec/acceptance/nodesets/centos-64-x64.yml @@ -0,0 +1,11 @@ +HOSTS: + centos-64-x64: + roles: + - master + platform: el-6-x86_64 + box : centos-64-x64-vbox4210-nocm + box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210-nocm.box + hypervisor : vagrant +CONFIG: + log_level: verbose + type: foss diff --git a/git/spec/acceptance/nodesets/default.yml b/git/spec/acceptance/nodesets/default.yml new file mode 100644 index 000000000..b392dab75 --- /dev/null +++ b/git/spec/acceptance/nodesets/default.yml @@ -0,0 +1,12 @@ +HOSTS: + ubuntu-server-12042-x64: + roles: + - master + platform: ubuntu-server-12.04-amd64 + box: ubuntu-server-12042-x64-vbox4210-nocm + box_url: http://puppet-vagrant-boxes.puppetlabs.com/ubuntu-server-12042-x64-vbox4210-nocm.box + hypervisor: vagrant + +CONFIG: + log_level: verbose + type: foss diff --git a/git/spec/acceptance/nodesets/ubuntu-server-12042-x64.yml b/git/spec/acceptance/nodesets/ubuntu-server-12042-x64.yml new file mode 100644 index 000000000..b392dab75 --- /dev/null +++ b/git/spec/acceptance/nodesets/ubuntu-server-12042-x64.yml @@ -0,0 +1,12 @@ +HOSTS: + ubuntu-server-12042-x64: + roles: + - master + platform: ubuntu-server-12.04-amd64 + box: ubuntu-server-12042-x64-vbox4210-nocm + box_url: http://puppet-vagrant-boxes.puppetlabs.com/ubuntu-server-12042-x64-vbox4210-nocm.box + hypervisor: vagrant + +CONFIG: + log_level: verbose + type: foss diff --git a/git/spec/classes/git_gitosis_spec.rb b/git/spec/classes/git_gitosis_spec.rb new file mode 100644 index 000000000..eb8ceada6 --- /dev/null +++ b/git/spec/classes/git_gitosis_spec.rb @@ -0,0 +1,10 @@ +require 'spec_helper' + +describe 'git::gitosis' do + + context 'defaults' do + it { should contain_package('gitosis') } + it { should contain_class('git') } + it { should create_class('git::gitosis') } + end +end diff --git a/git/spec/classes/git_spec.rb b/git/spec/classes/git_spec.rb new file mode 100644 index 000000000..fabdf5194 --- /dev/null +++ b/git/spec/classes/git_spec.rb @@ -0,0 +1,70 @@ +require 'spec_helper' + +describe 'git' do + + context 'defaults' do + it { should contain_package('git') } + end + + context 'with package_manage set to false' do + let(:params) { + { + :package_manage => false, + } + } + it { should_not contain_package('git') } + end + + context 'with a custom git package name' do + let(:params) { + { + :package_name => 'gitolite', + } + } + it { should contain_package('gitolite') } + end + + context 'with package_ensure => latest' do + let(:params) { + { + :package_ensure => 'latest', + } + } + it { should contain_package('git').with( + { + 'ensure' => 'latest' + } + )} + end + + context 'with configs' do + let(:params) { + { + :configs => { + "user.name" => {"value" => "test"}, + "user.email" => "test@example.com" + } + } + } + it { should contain_git__config('user.name') } + it { should contain_git__config('user.email') } + end + + context 'with configs and configs defaults' do + let(:params) { + { + :configs => { + "core.filemode" => false + }, + :configs_defaults => { + "scope" => "system" + } + } + } + it { should contain_git__config('core.filemode').with( + 'value' => false, + 'scope' => 'system' + ) } + end + +end diff --git a/git/spec/classes/git_subtree_spec.rb b/git/spec/classes/git_subtree_spec.rb new file mode 100644 index 000000000..137e59241 --- /dev/null +++ b/git/spec/classes/git_subtree_spec.rb @@ -0,0 +1,77 @@ +require 'spec_helper' + +describe 'git::subtree' do + + versions = ['1.6.0', '1.7.0', '1.7.11'] + versions.each do |version| + context "when git version is #{version}" do + let(:facts) { { + :git_version => version, + } } + if version < '1.7.0' + it 'should fail' do + expect { should create_class('git::subtree') }.to raise_error(Puppet::Error, /git-subtree requires git 1.7 or later!/) + end + else + it { should create_class('git::subtree') } + it { should contain_class('git') } + it { should contain_package('asciidoc') } + it { should contain_package('xmlto') } + it { should contain_exec('Install git-subtree') } + it { should contain_exec('Build git-subtree') } + + it { should create_file('/etc/bash_completion.d/git-subtree').with({ + :ensure => 'file', + :source => 'puppet:///modules/git/subtree/bash_completion.sh', + :mode => '0644', + })} + end + end + end + + context 'when git version > 1.7.0 and < 1.7.11' do + let(:facts) { { + :git_version => '1.7.0', + :git_exec_path => '/usr/lib/git-core', + :git_html_path => "/usr/share/doc/git" + } } + + it { should create_vcsrepo('/usr/src/git-subtree').with({ + :ensure => 'present', + :source => 'https://github.com/apenwarr/git-subtree.git', + :provider => 'git', + :revision => '2793ee6ba', + })} + + it { should create_exec('Build git-subtree').with({ + :command => 'make prefix=/usr libexecdir=/usr/lib/git-core', + :creates => '/usr/src/git-subtree/git-subtree', + :cwd => '/usr/src/git-subtree', + })} + + it { should create_exec('Install git-subtree').with({ + :command => 'make prefix=/usr libexecdir=/usr/lib/git-core install', + :cwd => '/usr/src/git-subtree', + })} + end + + context 'when git version >= 1.7.11' do + let(:facts) { { + :git_version => '1.7.11', + :git_exec_path => '/usr/lib/git-core', + :git_html_path => "/usr/share/doc/git" + } } + + it { should create_exec('Build git-subtree').with({ + :creates => '/usr/share/doc/git/contrib/subtree/git-subtree', + :cwd => '/usr/share/doc/git/contrib/subtree', + })} + + it { should create_exec('Install git-subtree').with({ + :command => 'make prefix=/usr libexecdir=/usr/lib/git-core install', + :cwd => '/usr/share/doc/git/contrib/subtree', + })} + end + + +end diff --git a/git/spec/defines/git_config_spec.rb b/git/spec/defines/git_config_spec.rb new file mode 100644 index 000000000..4149fc77a --- /dev/null +++ b/git/spec/defines/git_config_spec.rb @@ -0,0 +1,37 @@ +require 'spec_helper' + +describe 'git::config', :type => :define do + context 'has working default parameters' do + let(:title) { 'user.name' } + let(:params) { + { + :value => 'JC Denton', + } + } + it do + should contain_git_config('user.name').with( + 'value' => 'JC Denton', + 'key' => 'user.name', + 'user' => 'root' + ) + have_git_config_resource_count(1) + end + end + context 'allows you to change user' do + let(:title) { 'user.email' } + let(:params) { + { + :value => 'jcdenton@UNATCO.com', + :user => 'admin' + } + } + it do + should contain_git_config('user.email').with( + 'value' => 'jcdenton@UNATCO.com', + 'key' => 'user.email', + 'user' => 'admin' + ) + have_git_config_resource_count(1) + end + end +end diff --git a/git/spec/spec_helper.rb b/git/spec/spec_helper.rb new file mode 100644 index 000000000..2c6f56649 --- /dev/null +++ b/git/spec/spec_helper.rb @@ -0,0 +1 @@ +require 'puppetlabs_spec_helper/module_spec_helper' diff --git a/git/spec/spec_helper_acceptance.rb b/git/spec/spec_helper_acceptance.rb new file mode 100644 index 000000000..de3635102 --- /dev/null +++ b/git/spec/spec_helper_acceptance.rb @@ -0,0 +1,24 @@ +require 'beaker-rspec/spec_helper' +require 'beaker-rspec/helpers/serverspec' + +hosts.each do |host| + # Install Puppet + install_puppet +end + +RSpec.configure do |c| + # Project root + proj_root = File.expand_path(File.join(File.dirname(__FILE__), '..')) + + # Readable test descriptions + c.formatter = :documentation + + # Configure all nodes in nodeset + c.before :suite do + # Install module and dependencies + puppet_module_install(:source => proj_root, :module_name => 'git') + hosts.each do |host| + on host, puppet('module', 'install', 'puppetlabs-vcsrepo'), { :acceptable_exit_codes => [0,1] } + end + end +end \ No newline at end of file diff --git a/git/spec/unit/facter/git_exec_path_spec.rb b/git/spec/unit/facter/git_exec_path_spec.rb new file mode 100644 index 000000000..0394869a0 --- /dev/null +++ b/git/spec/unit/facter/git_exec_path_spec.rb @@ -0,0 +1,27 @@ +require "spec_helper" + +describe Facter::Util::Fact do + before { + Facter.clear + } + + describe "git_exec_path" do + + context 'windows' do + it do + Facter.fact(:osfamily).stubs(:value).returns('windows') + Facter::Util::Resolution.expects(:exec).with("git --exec-path 2>nul").returns('windows_path_change') + Facter.fact(:git_exec_path).value.should == 'windows_path_change' + end + end + + context 'non-windows' do + it do + Facter.fact(:osfamily).stubs(:value).returns('RedHat') + Facter::Util::Resolution.expects(:exec).with("git --exec-path 2>/dev/null").returns('/usr/libexec/git-core') + Facter.fact(:git_exec_path).value.should == '/usr/libexec/git-core' + end + end + + end +end \ No newline at end of file diff --git a/git/spec/unit/facter/git_html_path_spec.rb b/git/spec/unit/facter/git_html_path_spec.rb new file mode 100644 index 000000000..4f332e248 --- /dev/null +++ b/git/spec/unit/facter/git_html_path_spec.rb @@ -0,0 +1,27 @@ +require "spec_helper" + +describe Facter::Util::Fact do + before { + Facter.clear + } + + describe "git_html_path" do + + context 'windows' do + it do + Facter.fact(:osfamily).stubs(:value).returns('windows') + Facter::Util::Resolution.expects(:exec).with("git --html-path 2>nul").returns('windows_path_change') + Facter.fact(:git_html_path).value.should == 'windows_path_change' + end + end + + context 'non-windows' do + it do + Facter.fact(:osfamily).stubs(:value).returns('RedHat') + Facter::Util::Resolution.expects(:exec).with("git --html-path 2>/dev/null").returns('/usr/share/doc/git-1.7.1') + Facter.fact(:git_html_path).value.should == '/usr/share/doc/git-1.7.1' + end + end + + end +end \ No newline at end of file diff --git a/git/spec/unit/facter/git_version_spec.rb b/git/spec/unit/facter/git_version_spec.rb new file mode 100644 index 000000000..e12057700 --- /dev/null +++ b/git/spec/unit/facter/git_version_spec.rb @@ -0,0 +1,35 @@ +require "spec_helper" + +describe Facter::Util::Fact do + before { + Facter.clear + } + + describe "git_version" do + context 'vanilla git' do + it do + git_version_output = 'git version 2.1.2' + Facter::Util::Resolution.expects(:exec).with("git --version 2>&1").returns(git_version_output) + Facter.value(:git_version).should == "2.1.2" + end + end + + context 'git with hub' do + it do + git_version_output = <<-EOS +git version 2.1.2 +hub version 1.12.2 + EOS + Facter::Util::Resolution.expects(:exec).with("git --version 2>&1").returns(git_version_output) + Facter.value(:git_version).should == "2.1.2" + end + end + + context 'no git present' do + it do + Facter::Util::Resolution.expects(:which).with("git").returns(false) + Facter.value(:git_version).should be_nil + end + end + end +end diff --git a/git/spec/unit/puppet/provider/git_config/git_config_spec.rb b/git/spec/unit/puppet/provider/git_config/git_config_spec.rb new file mode 100644 index 000000000..6f60390bf --- /dev/null +++ b/git/spec/unit/puppet/provider/git_config/git_config_spec.rb @@ -0,0 +1,14 @@ +require 'spec_helper' + +describe Puppet::Type.type(:git_config).provider(:git_config) do + + let(:resource) { Puppet::Type.type(:git_config).new( + { + :key => 'user.email', + :value => 'john.doe@example.com', + } + )} + + let(:provider) { resource.provider } + +end \ No newline at end of file diff --git a/git/spec/unit/puppet/type/git_config/git_config_spec.rb b/git/spec/unit/puppet/type/git_config/git_config_spec.rb new file mode 100644 index 000000000..0fd26856d --- /dev/null +++ b/git/spec/unit/puppet/type/git_config/git_config_spec.rb @@ -0,0 +1,49 @@ +#!/usr/bin/env ruby + +require 'spec_helper' + +describe Puppet::Type.type(:git_config) do + + before do + @class = described_class + @provider_class = @class.provide(:fake) { mk_resource_methods } + @provider = @provider_class.new + @resource = stub 'resource', :resource => nil, :provider => @provider + + @class.stubs(:defaultprovider).returns @provider_class + @class.any_instance.stubs(:provider).returns @provider + end + + it "should have :name as its keyattribute" do + @class.key_attributes.should == [:name] + end + + describe "when validating attributes" do + + params = [ + :name, + :user, + :key, + :section, + :scope, + ] + + properties = [ + :value, + ] + + params.each do |param| + it "should have a #{param} parameter" do + @class.attrtype(param).should == :param + end + end + + properties.each do |property| + it "should have a #{property} property" do + @class.attrtype(property).should == :property + end + end + + end + +end \ No newline at end of file diff --git a/git/tests/gitosis.pp b/git/tests/gitosis.pp new file mode 100644 index 000000000..e6240ae21 --- /dev/null +++ b/git/tests/gitosis.pp @@ -0,0 +1 @@ +class { 'git::gitosis': } diff --git a/git/tests/init.pp b/git/tests/init.pp new file mode 100644 index 000000000..c23290471 --- /dev/null +++ b/git/tests/init.pp @@ -0,0 +1 @@ +class { 'git': } From ef47a3076cd891278dea6f7f21904c09cfe9d24b Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Thu, 7 Jan 2016 11:41:07 -0500 Subject: [PATCH 12/35] Update fluentd to ddc5f0e4c6c53d15f0cbd34f74bfaa91a0fb299f ddc5f0e4c6c53d15f0cbd34f74bfaa91a0fb299f Merge pull request #1 from EmilienM/file/fullpath 31fde18d4910a87f4e54e58501361df1364e74f0 Manage td-agent.conf file with a fully qualified path Change-Id: I9c27ddaf62f803d6097821a8458cd18805943b4e --- Puppetfile | 2 +- fluentd/manifests/install.pp | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Puppetfile b/Puppetfile index f227a6ce0..cd9ccd5b8 100644 --- a/Puppetfile +++ b/Puppetfile @@ -59,7 +59,7 @@ mod 'firewall', :git => 'https://github.com/puppetlabs/puppetlabs-firewall.git' mod 'fluentd', - :commit => 'b462da7d1c6290afba38fd7e64226990ecf795c7', + :commit => 'ddc5f0e4c6c53d15f0cbd34f74bfaa91a0fb299f', :git => 'https://github.com/soylent/konstantin-fluentd.git' mod 'galera', diff --git a/fluentd/manifests/install.pp b/fluentd/manifests/install.pp index e5e859774..df95821f5 100644 --- a/fluentd/manifests/install.pp +++ b/fluentd/manifests/install.pp @@ -12,7 +12,7 @@ } -> file { $fluentd::config_file: - ensure => present, - content => file('fluentd/td-agent.conf'), + ensure => present, + source => 'puppet:///modules/fluentd/td-agent.conf', } } From 392f6bdbe6dc5f106336e7f133aad734e3eb736f Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Thu, 7 Jan 2016 11:49:36 -0500 Subject: [PATCH 13/35] Sync to latest stable/liberty upstream. Update aodh to 5c5478104458d7e682df71036d6af2d571d21249 5c5478104458d7e682df71036d6af2d571d21249 Support MongoDB as backend db aaa5fd2909c628ac588f65ebcd429a71d010a3b9 unit tests: make Keystone_endpoint match service by name/type Change-Id: I3779e7cc913bdd955e55315bd31459c2033a9738 Update ceilometer to f27e53e9903d847fb5c14685d8ff009319c7ea8f f27e53e9903d847fb5c14685d8ff009319c7ea8f unit tests: make Keystone_endpoint match service by name/type Change-Id: I2e4bd6844624ef8ed07194378f662fdcfacee54c Update cinder to 40d5f3e385a54e7626fc14cabccaec6eebcad66b 40d5f3e385a54e7626fc14cabccaec6eebcad66b Support of PyMySQL driver for MySQL backend 2bb26b36035423ee10ee093d8c11776c75261d28 unit tests: make Keystone_endpoint match service by name/type 54cd5fdee50e05dab937ecac1c52588cccbf82c1 Fix multiple rbd backend with multiple users Change-Id: If6c921f888baa23b7c905173d104fa447c69f62e Update glance to 4c7c407fbd15977d81ca4df6309ad1a839ad59e1 4c7c407fbd15977d81ca4df6309ad1a839ad59e1 Stop collecting glance services by name and Make Keystone_endpoint match service by name/type Change-Id: I02bb2a3ad5fe21565a1e04f7b6fb2d9a47461b61 Update gnocchi to e774f7df73ccce96ff2fe256e40a9de078b0fffb e774f7df73ccce96ff2fe256e40a9de078b0fffb Merge "Implement ::gnocchi::client" into stable/liberty 9fb5466315677b6b748aa1be81046ddcc08bc3fc Implement ::gnocchi::client aafe2ce5211e0b1e922fab9c18d24ff4eb6361d4 Fix ::gnocchi::statsd class parameters documentation 0c3ec1cb6abc1d353f0297b5a381b192729b7d73 unit tests: make Keystone_endpoint match service by name/type Change-Id: I2548f006e35a95d8d0fdacb9a2da2f069c59e2fd Update heat to 408dfb4d65992463d62fc7bafe8e1fc6b4f29a60 408dfb4d65992463d62fc7bafe8e1fc6b4f29a60 Support of PyMySQL driver for MySQL backend 4b56b0861deeb2857c9623536a59964886aa3885 unit tests: make Keystone_endpoint match service by name/type Change-Id: I6174809f48eaf59295f0d12e3ffd98ee94ee9270 Update ironic to 342cc10fb080a787d971f39df0901750996807d3 342cc10fb080a787d971f39df0901750996807d3 Do not use notify for db-sync from main init class 23975a0d83ea90748ec83d94197c1ee9974ca2b2 Make Keystone_endpoint match service by name/type Change-Id: I1fb5484724014b1088824cf0c569b49573722b3f Update keystone to def536208d70b70171a8f6b5797077b0638abde9 def536208d70b70171a8f6b5797077b0638abde9 Manage Keystone_endpoint and Keystone_service without warnings by default 3c65cfd07239299776f39617e5e6b45541385e55 Undeprecate the $keystone::endpoint::version param Change-Id: Ic24ebbf65aa6aef957a1de2e72d60c6d0e6d528e Update manila to 9d74004a388f65d8e54c8ab4dfb5832a3b024925 9d74004a388f65d8e54c8ab4dfb5832a3b024925 Support of PyMySQL driver for MySQL backend 5b69dadeb47164767a795ecacf7aa0587001119a unit tests: make Keystone_endpoint match service by name/type Change-Id: Ia6c7f1891b4c17ad4b739cf3ffe558ebdfb57922 Update neutron to 3908534c2d05a5040f6adf436f8f96a22be528bb 3908534c2d05a5040f6adf436f8f96a22be528bb Support of PyMySQL driver for MySQL backend 02bb1a14b0641f62856ac86750cc835d6ec77e42 fwaas config needs to notify l3 agent service 2e4fe1eb6b4b1d5ae3bc582b694ff94fe96a84da Specify search path used for command execution c587fd138de987cc061b9f810f59d14bc02f4189 Merge "Adds configuration support for OpenDaylight SDN Controller" into stable/liberty 1621e84caeb83cd86a632aaa284f344289b9cdd7 Make Keystone_endpoint match service by name/type e3ada8a858e9abc0535d615d04afed92edd93c4e Adds configuration support for OpenDaylight SDN Controller Change-Id: I4cf42a14aac862a21cf4cea1a3250a0944832c82 Update nova to 972c4d6e76a539c59b2df78b25ca03859f2144bb 972c4d6e76a539c59b2df78b25ca03859f2144bb Support of PyMySQL driver for MySQL backend a697c9ecd4b86f579231316604c60ec39b9a6916 Make Keystone_endpoint match service by name/type Change-Id: Ib04f061bb59c63f6c5cdfbcec7c85d2c6c417900 Update sahara to 64774e9cf344f897163b49afcb725ccc62689610 64774e9cf344f897163b49afcb725ccc62689610 unit tests: make Keystone_endpoint match service by name/type Change-Id: I7a2c92ec12c3c59529299af6ca8504efecc9321f Update swift to 8c9c77394e6b294c047c200d740f41d08eeab057 8c9c77394e6b294c047c200d740f41d08eeab057 Merge "Use include for swift::storage::${type}" into stable/liberty 48621cc2868aebf3279956324366ebd0a30d2e1b Merge "Define ${name}-auditors in storage/generic.pp" into stable/liberty 80af0fe5fb48aeda2bd71e19ed05871c2de24732 Merge "swiftinit provider check init/systemd file content" into stable/liberty bfad519f9dfb45018506d3a091677a410486d06c Merge "Manage swift with swiftinit service provider" into stable/liberty 00f9620f58b2a1dc4c1039dc1b2052e91b33732d Merge "Update ceilometer proxy template" into stable/liberty 64eabe01951f215f0eba5faf654ec6ce23a4eaea Merge "Require swift package installation before swift user." into stable/liberty 1d641ba2f3ebce5dfe5ba7636f21610f3d6d7969 Update ceilometer proxy template 3ba578fba7666f20db5b53803ca52c5b9239f863 Use include for swift::storage::${type} d38e8e64767f91902232f98ca4c129a4efde6579 Define ${name}-auditors in storage/generic.pp f9805317220b4a7dad6562e9e99149f284f7f698 swiftinit provider check init/systemd file content c1d7ddc08bab9a8de02756d76b21d91165066676 Manage swift with swiftinit service provider 8102ffa5fb344f45606af39509b232b5ee354704 Require swift package installation before swift user. 0a259821207ebfd67d5adbd057309448e778b670 swift-ring-builder 2.2.2+ output parse fix 84328a7eab950501558294467ea190ac3f6f499a unit tests: make Keystone_endpoint match service by name/type Change-Id: I03e06264420d4570b01a48e9741e6f54ee8160e1 Update tripleo to 8ebfc67734b6525f9a03aad9617d367d68780dac 8ebfc67734b6525f9a03aad9617d367d68780dac Merge "Adding psych on Gemfile explicitly" into stable/liberty 2ae89722af5fd1cda549ac0c65ec3df3f6d78fb0 Adds IPv6 support for interface_for_ip function d2f6f1529193ee657733fd0cba994d71355641f0 Merge "Set tunnel timeout for nova_novncproxy" into stable/liberty 828dab5936f0c57eb04e41148df9ba48ed613632 Merge "Modify cassandra dependency" into stable/liberty 567aeecc5c2ff7f8e04082deb5e88a7abaca0e80 Merge "Adding MidoNet LoadBalancing options" into stable/liberty b5d5ded94e30fa07fd96c8a3b3e47f911ab4df31 Merge "Fix unit tests failing against Puppet 4.3.0" into stable/liberty fea19cd3e2dfa6143a331d04a992f94746c1d30c Modify cassandra dependency e103bfa4614bbd8776f84aad3e7fc924c27043fe Adding psych on Gemfile explicitly 18815adc8e0c402f053525f2aae7fb3a2a27eb8b Adding MidoNet LoadBalancing options 5307a1df127ecdb6b245f3c5d11ec7e3951b85e2 Remove `validate_ip_address` validator. e6b55f2894b93ea4ae8193d78f806ee3c88650b1 MidoNet services manifests 7974a613d27457226d803be376b524596cac7d9a Fix unit tests failing against Puppet 4.3.0 469e5a564aece7bbf8f2de92a45c5422d84bdf60 Set tunnel timeout for nova_novncproxy Change-Id: I75654fb1251ea4dad307dd95e635ef4ddd71b336 Update trove to e2e6b767f82989ce6e38b77c2f33b6fe7efe1b6d e2e6b767f82989ce6e38b77c2f33b6fe7efe1b6d Make Keystone_endpoint match service by name/type Change-Id: I945a8470bf76f0e8828fa112cc4e75a4a46843d5 Update openstacklib to 247553cf382083fb85ca7987ee7d3cf208ec3e64 247553cf382083fb85ca7987ee7d3cf208ec3e64 Update os_database_connection with extra param Change-Id: I6012c63f373a9729260a65c350e53f051686a53b --- Puppetfile | 32 +-- aodh/manifests/db.pp | 5 +- aodh/manifests/params.pp | 2 + aodh/spec/classes/aodh_db_spec.rb | 16 ++ aodh/spec/classes/aodh_keystone_auth_spec.rb | 21 +- .../classes/ceilometer_keystone_auth_spec.rb | 31 ++- cinder/manifests/backend/rbd.pp | 7 +- cinder/manifests/db.pp | 10 +- cinder/manifests/params.pp | 2 + cinder/spec/acceptance/basic_cinder_spec.rb | 2 +- cinder/spec/classes/cinder_db_spec.rb | 43 +++- .../spec/classes/cinder_keystone_auth_spec.rb | 36 ++- cinder/spec/classes/cinder_volume_rbd_spec.rb | 5 +- .../spec/defines/cinder_backend_rbd_spec.rb | 8 +- glance/manifests/keystone/auth.pp | 8 +- glance/spec/acceptance/basic_glance_spec.rb | 27 ++- .../spec/classes/glance_keystone_auth_spec.rb | 26 +-- gnocchi/examples/site.pp | 2 + gnocchi/manifests/client.pp | 21 ++ gnocchi/manifests/params.pp | 2 + gnocchi/manifests/statsd.pp | 28 +-- gnocchi/spec/acceptance/basic_gnocchi_spec.rb | 1 + gnocchi/spec/classes/gnocchi_client_spec.rb | 33 +++ .../classes/gnocchi_keystone_auth_spec.rb | 21 +- heat/examples/site.pp | 2 +- heat/manifests/db.pp | 11 +- heat/manifests/params.pp | 2 + heat/spec/acceptance/basic_heat_spec.rb | 2 +- heat/spec/classes/heat_db_spec.rb | 36 ++- heat/spec/classes/heat_init_spec.rb | 2 +- .../classes/heat_keystone_auth_cfn_spec.rb | 19 +- heat/spec/classes/heat_keystone_auth_spec.rb | 19 +- ironic/manifests/init.pp | 1 - ironic/manifests/keystone/auth.pp | 2 +- .../spec/classes/ironic_keystone_auth_spec.rb | 23 +- keystone/manifests/endpoint.pp | 26 ++- .../manifests/resource/service_identity.pp | 33 ++- .../spec/classes/keystone_endpoint_spec.rb | 11 +- ...keystone_resource_service_identity_spec.rb | 18 +- manila/manifests/db.pp | 12 +- manila/manifests/params.pp | 2 + manila/spec/acceptance/basic_manila_spec.rb | 2 +- manila/spec/classes/manila_db_spec.rb | 36 ++- .../spec/classes/manila_keystone_auth_spec.rb | 20 +- manila/spec/classes/manila_spec.rb | 4 +- neutron/manifests/db.pp | 12 +- neutron/manifests/keystone/auth.pp | 2 +- neutron/manifests/params.pp | 2 + neutron/manifests/plugins/ml2/opendaylight.pp | 60 +++++ neutron/manifests/plugins/ovs/opendaylight.pp | 79 +++++++ neutron/manifests/services/fwaas.pp | 3 + neutron/spec/acceptance/basic_neutron_spec.rb | 2 +- neutron/spec/classes/neutron_db_spec.rb | 38 +++- .../classes/neutron_keystone_auth_spec.rb | 27 +-- .../neutron_plugins_ml2_opendaylight_spec.rb | 73 ++++++ .../neutron_plugins_ovs_opendaylight_spec.rb | 88 ++++++++ .../classes/neutron_services_fwaas_spec.rb | 3 +- nova/manifests/db.pp | 10 +- nova/manifests/keystone/auth.pp | 2 +- nova/manifests/params.pp | 2 + nova/spec/acceptance/basic_nova_spec.rb | 2 +- nova/spec/classes/nova_db_spec.rb | 32 ++- nova/spec/classes/nova_keystone_auth_spec.rb | 73 +++--- .../functions/os_database_connection.rb | 15 +- .../functions/os_database_connection_spec.rb | 45 +++- .../spec/classes/sahara_keystone_auth_spec.rb | 11 +- swift/README.md | 86 ++++++- swift/files/swift-account.conf.upstart | 23 -- swift/files/swift-container-sync.conf.upstart | 20 -- swift/files/swift-container.conf.upstart | 23 -- swift/files/swift-object.conf.upstart | 23 -- .../lib/puppet/provider/service/swiftinit.rb | 210 ++++++++++++++++++ .../lib/puppet/provider/swift_ring_builder.rb | 22 ++ swift/manifests/init.pp | 3 +- swift/manifests/objectexpirer.pp | 27 ++- swift/manifests/params.pp | 35 ++- swift/manifests/proxy.pp | 31 ++- swift/manifests/proxy/ceilometer.pp | 55 ++++- swift/manifests/service.pp | 71 ++++++ swift/manifests/storage/account.pp | 61 ++--- swift/manifests/storage/container.pp | 75 ++++--- swift/manifests/storage/generic.pp | 78 ++++--- swift/manifests/storage/object.pp | 61 ++--- swift/manifests/storage/server.pp | 15 +- swift/spec/acceptance/basic_swift_spec.rb | 87 ++++++++ .../spec/classes/swift_keystone_auth_spec.rb | 26 +-- .../spec/classes/swift_objectexpirer_spec.rb | 26 +++ .../classes/swift_proxy_ceilometer_spec.rb | 40 +++- swift/spec/classes/swift_proxy_spec.rb | 86 ++++--- .../classes/swift_storage_account_spec.rb | 49 +++- swift/spec/classes/swift_storage_all_spec.rb | 4 +- .../classes/swift_storage_container_spec.rb | 71 +++--- .../spec/classes/swift_storage_object_spec.rb | 56 ++++- .../defines/swift_storage_generic_spec.rb | 194 ++++++++++++---- .../spec/defines/swift_storage_server_spec.rb | 12 +- .../puppet/provider/service/swiftinit_spec.rb | 71 ++++++ .../provider/swift_ring_builder_spec.rb | 39 ++++ swift/templates/proxy/ceilometer.conf.erb | 17 +- tripleo/.fixtures.yml | 21 ++ tripleo/Gemfile | 4 + tripleo/lib/facter/netmask_ipv6.rb | 47 ++++ .../lib/puppet/parser/functions/extract_id.rb | 14 ++ .../parser/functions/interface_for_ip.rb | 32 +-- .../functions/list_to_zookeeper_hash.rb | 24 ++ tripleo/manifests/cluster/cassandra.pp | 68 ++++++ tripleo/manifests/cluster/zookeeper.pp | 65 ++++++ tripleo/manifests/loadbalancer.pp | 25 +++ tripleo/manifests/network/midonet/agent.pp | 61 +++++ tripleo/manifests/network/midonet/api.pp | 117 ++++++++++ .../classes/tripleo_cluster_cassandra_spec.rb | 54 +++++ .../classes/tripleo_cluster_zookeeper_spec.rb | 115 ++++++++++ tripleo/spec/classes/tripleo_firewall_spec.rb | 5 +- .../classes/tripleo_midonet_agent_spec.rb | 61 +++++ .../spec/classes/tripleo_midonet_api_spec.rb | 72 ++++++ tripleo/spec/spec_helper.rb | 4 + trove/manifests/keystone/auth.pp | 2 +- trove/manifests/taskmanager.pp | 3 +- .../spec/classes/trove_keystone_auth_spec.rb | 19 +- 118 files changed, 3021 insertions(+), 742 deletions(-) create mode 100644 gnocchi/manifests/client.pp create mode 100644 gnocchi/spec/classes/gnocchi_client_spec.rb create mode 100644 neutron/manifests/plugins/ml2/opendaylight.pp create mode 100644 neutron/manifests/plugins/ovs/opendaylight.pp create mode 100644 neutron/spec/classes/neutron_plugins_ml2_opendaylight_spec.rb create mode 100644 neutron/spec/classes/neutron_plugins_ovs_opendaylight_spec.rb delete mode 100644 swift/files/swift-account.conf.upstart delete mode 100644 swift/files/swift-container-sync.conf.upstart delete mode 100644 swift/files/swift-container.conf.upstart delete mode 100644 swift/files/swift-object.conf.upstart create mode 100644 swift/lib/puppet/provider/service/swiftinit.rb create mode 100644 swift/manifests/service.pp create mode 100644 swift/spec/unit/puppet/provider/service/swiftinit_spec.rb create mode 100644 tripleo/lib/facter/netmask_ipv6.rb create mode 100644 tripleo/lib/puppet/parser/functions/extract_id.rb create mode 100644 tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb create mode 100644 tripleo/manifests/cluster/cassandra.pp create mode 100644 tripleo/manifests/cluster/zookeeper.pp create mode 100644 tripleo/manifests/network/midonet/agent.pp create mode 100644 tripleo/manifests/network/midonet/api.pp create mode 100644 tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb create mode 100644 tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb create mode 100644 tripleo/spec/classes/tripleo_midonet_agent_spec.rb create mode 100644 tripleo/spec/classes/tripleo_midonet_api_spec.rb diff --git a/Puppetfile b/Puppetfile index cd9ccd5b8..0a0a3fe0b 100644 --- a/Puppetfile +++ b/Puppetfile @@ -1,5 +1,5 @@ mod 'aodh', - :commit => 'a09301ab15b621e5e485fb605ab488e64a059d2a', + :commit => '5c5478104458d7e682df71036d6af2d571d21249', :git => 'https://github.com/openstack/puppet-aodh.git' mod 'apache', @@ -15,7 +15,7 @@ mod 'cassandra', :git => 'https://github.com/locp/cassandra.git' mod 'ceilometer', - :commit => 'b9434a8d9d35300fb8247b862823bf6acbde15aa', + :commit => 'f27e53e9903d847fb5c14685d8ff009319c7ea8f', :git => 'https://github.com/openstack/puppet-ceilometer.git' mod 'ceph', @@ -27,7 +27,7 @@ mod 'certmonger', :git => 'https://github.com/rcritten/puppet-certmonger.git' mod 'cinder', - :commit => '5add15bc80b8867b3d079f8aa563a609800f3e61', + :commit => '40d5f3e385a54e7626fc14cabccaec6eebcad66b', :git => 'https://github.com/openstack/puppet-cinder.git' mod 'common', @@ -71,7 +71,7 @@ mod 'git', :git => 'https://github.com/puppetlabs/puppetlabs-git.git' mod 'glance', - :commit => 'f4495d93a064ea8d1f5a023c156ab73e14e69198', + :commit => '4c7c407fbd15977d81ca4df6309ad1a839ad59e1', :git => 'https://github.com/openstack/puppet-glance.git' mod 'gluster', @@ -79,7 +79,7 @@ mod 'gluster', :git => 'https://github.com/purpleidea/puppet-gluster.git' mod 'gnocchi', - :commit => 'f658ca8213fa94ff9de5abee73720c271631bd4d', + :commit => 'e774f7df73ccce96ff2fe256e40a9de078b0fffb', :git => 'https://github.com/openstack/puppet-gnocchi.git' mod 'haproxy', @@ -87,7 +87,7 @@ mod 'haproxy', :git => 'https://github.com/puppetlabs/puppetlabs-haproxy.git' mod 'heat', - :commit => 'fac1ae0e794c2f1ba66132e77f83f9e8f035dcdf', + :commit => '408dfb4d65992463d62fc7bafe8e1fc6b4f29a60', :git => 'https://github.com/openstack/puppet-heat.git' mod 'horizon', @@ -103,7 +103,7 @@ mod 'ipa', :git => 'https://github.com/xbezdick/puppet-ipa-1.git' mod 'ironic', - :commit => 'ec08404c8c23f499b972e6348f0bf6c503dc7f3e', + :commit => '342cc10fb080a787d971f39df0901750996807d3', :git => 'https://github.com/openstack/puppet-ironic.git' mod 'java', @@ -119,7 +119,7 @@ mod 'keepalived', :git => 'https://github.com/Unyonsys/puppet-module-keepalived.git' mod 'keystone', - :commit => 'bee02643562f38af2c5b2ebb4ba65c0e86388952', + :commit => 'def536208d70b70171a8f6b5797077b0638abde9', :git => 'https://github.com/openstack/puppet-keystone.git' mod 'kibana3', @@ -127,7 +127,7 @@ mod 'kibana3', :git => 'https://github.com/thejandroman/puppet-kibana3.git' mod 'manila', - :commit => '068aa077b8698f900fec3a2e5a1690214b5acfd2', + :commit => '9d74004a388f65d8e54c8ab4dfb5832a3b024925', :git => 'https://github.com/openstack/puppet-manila.git' mod 'memcached', @@ -163,11 +163,11 @@ mod 'nagios', :git => 'https://github.com/gildub/puppet-nagios-openstack.git' mod 'neutron', - :commit => '2b427bb1f9d0b79c49a536ab6046f0095cd6995c', + :commit => '3908534c2d05a5040f6adf436f8f96a22be528bb', :git => 'https://github.com/openstack/puppet-neutron.git' mod 'nova', - :commit => 'a495b4010d69adcc36927c95036b7d854a785700', + :commit => '972c4d6e76a539c59b2df78b25ca03859f2144bb', :git => 'https://github.com/openstack/puppet-nova.git' mod 'nssdb', @@ -187,7 +187,7 @@ mod 'openstack_extras', :git => 'https://github.com/openstack/puppet-openstack_extras.git' mod 'openstacklib', - :commit => '142f9d1be907c45115ea4d92277278a682fb53f7', + :commit => '247553cf382083fb85ca7987ee7d3cf208ec3e64', :git => 'https://github.com/openstack/puppet-openstacklib.git' mod 'pacemaker', @@ -219,7 +219,7 @@ mod 'rsync', :git => 'https://github.com/puppetlabs/puppetlabs-rsync.git' mod 'sahara', - :commit => '811a5e9e95ade9a3d8c741e8cbfdc4e57eaca503', + :commit => '64774e9cf344f897163b49afcb725ccc62689610', :git => 'https://github.com/openstack/puppet-sahara.git' mod 'sensu', @@ -243,7 +243,7 @@ mod 'stdlib', :git => 'https://github.com/puppetlabs/puppetlabs-stdlib.git' mod 'swift', - :commit => '5ca8b0b75f89e31be24d6ec45dfcb2bb38a93bdb', + :commit => '8c9c77394e6b294c047c200d740f41d08eeab057', :git => 'https://github.com/openstack/puppet-swift.git' mod 'sysctl', @@ -263,11 +263,11 @@ mod 'tomcat', :git => 'https://github.com/puppetlabs/puppetlabs-tomcat.git' mod 'tripleo', - :commit => 'aaef6b770e4af987400bb8c674304e5b47b30982', + :commit => '8ebfc67734b6525f9a03aad9617d367d68780dac', :git => 'https://github.com/openstack/puppet-tripleo.git' mod 'trove', - :commit => 'ded110e944204aad0cad7480e445cc241a138f14', + :commit => 'e2e6b767f82989ce6e38b77c2f33b6fe7efe1b6d', :git => 'https://github.com/openstack/puppet-trove' mod 'tuskar', diff --git a/aodh/manifests/db.pp b/aodh/manifests/db.pp index 350c84a8e..a5ceba4a3 100644 --- a/aodh/manifests/db.pp +++ b/aodh/manifests/db.pp @@ -54,7 +54,7 @@ $database_max_overflow_real = pick($::aodh::database_max_overflow, $database_max_overflow) validate_re($database_connection_real, - '(sqlite|mysql|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') + '(sqlite|mysql|postgresql|mongodb):\/\/(\S+:\S+@\S+\/\S+)?') if $database_connection_real { case $database_connection_real { @@ -67,6 +67,9 @@ $backend_package = false require 'postgresql::lib::python' } + /^mongodb:\/\//: { + $backend_package = $::aodh::params::pymongo_package_name + } /^sqlite:\/\//: { $backend_package = $::aodh::params::sqlite_package_name } diff --git a/aodh/manifests/params.pp b/aodh/manifests/params.pp index e769cf789..2bb273d99 100644 --- a/aodh/manifests/params.pp +++ b/aodh/manifests/params.pp @@ -19,6 +19,7 @@ $expirer_package_serice = 'openstack-aodh-expirer' $listener_package_name = 'openstack-aodh-listener' $listener_service_name = 'openstack-aodh-listener' + $pymongo_package_name = 'python-pymongo' $aodh_wsgi_script_path = '/var/www/cgi-bin/aodh' $aodh_wsgi_script_source = '/usr/lib/python2.7/site-packages/aodh/api/app.wsgi' } @@ -36,6 +37,7 @@ $expirer_package_serice = 'aodh-expirer' $listener_package_name = 'aodh-listener' $listener_service_name = 'aodh-listener' + $pymongo_package_name = 'python-pymongo' $aodh_wsgi_script_path = '/usr/lib/cgi-bin/aodh' $aodh_wsgi_script_source = '/usr/share/aodh-common/app.wsgi' } diff --git a/aodh/spec/classes/aodh_db_spec.rb b/aodh/spec/classes/aodh_db_spec.rb index 2b0b343c1..26e030230 100644 --- a/aodh/spec/classes/aodh_db_spec.rb +++ b/aodh/spec/classes/aodh_db_spec.rb @@ -45,6 +45,22 @@ end + context 'with mongodb backend' do + let :params do + { :database_connection => 'mongodb://localhost:1234/aodh', } + end + + it 'installs python-mongodb package' do + is_expected.to contain_package('aodh-backend-package').with( + :ensure => 'present', + :name => 'python-pymongo', + :tag => 'openstack' + ) + is_expected.to contain_aodh_config('database/connection').with_value('mongodb://localhost:1234/aodh') + is_expected.to contain_aodh_config('database/connection').with_value( params[:database_connection] ).with_secret(true) + end + + end context 'with incorrect database_connection string' do let :params do diff --git a/aodh/spec/classes/aodh_keystone_auth_spec.rb b/aodh/spec/classes/aodh_keystone_auth_spec.rb index 8e5cbe544..457f7f03a 100644 --- a/aodh/spec/classes/aodh_keystone_auth_spec.rb +++ b/aodh/spec/classes/aodh_keystone_auth_spec.rb @@ -26,13 +26,12 @@ :roles => ['admin'] )} - it { is_expected.to contain_keystone_service('aodh').with( + it { is_expected.to contain_keystone_service('aodh::alarming').with( :ensure => 'present', - :type => 'alarming', :description => 'OpenStack Alarming Service' ) } - it { is_expected.to contain_keystone_endpoint('RegionOne/aodh').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/aodh::alarming').with( :ensure => 'present', :public_url => 'http://127.0.0.1:8042', :admin_url => 'http://127.0.0.1:8042', @@ -48,7 +47,7 @@ :admin_url => 'http://10.10.10.12:81' } end - it { is_expected.to contain_keystone_endpoint('RegionOne/aodh').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/aodh::alarming').with( :ensure => 'present', :public_url => 'https://10.10.10.10:80', :internal_url => 'http://10.10.10.11:81', @@ -64,8 +63,8 @@ it { is_expected.to contain_keystone_user('aodhany') } it { is_expected.to contain_keystone_user_role('aodhany@services') } - it { is_expected.to contain_keystone_service('aodhany') } - it { is_expected.to contain_keystone_endpoint('RegionOne/aodhany') } + it { is_expected.to contain_keystone_service('aodhany::alarming') } + it { is_expected.to contain_keystone_endpoint('RegionOne/aodhany::alarming') } end describe 'when overriding service name' do @@ -77,8 +76,8 @@ it { is_expected.to contain_keystone_user('aodh') } it { is_expected.to contain_keystone_user_role('aodh@services') } - it { is_expected.to contain_keystone_service('aodh_service') } - it { is_expected.to contain_keystone_endpoint('RegionOne/aodh_service') } + it { is_expected.to contain_keystone_service('aodh_service::alarming') } + it { is_expected.to contain_keystone_endpoint('RegionOne/aodh_service::alarming') } end describe 'when disabling user configuration' do @@ -92,9 +91,8 @@ it { is_expected.not_to contain_keystone_user('aodh') } it { is_expected.to contain_keystone_user_role('aodh@services') } - it { is_expected.to contain_keystone_service('aodh').with( + it { is_expected.to contain_keystone_service('aodh::alarming').with( :ensure => 'present', - :type => 'alarming', :description => 'OpenStack Alarming Service' ) } @@ -112,9 +110,8 @@ it { is_expected.not_to contain_keystone_user('aodh') } it { is_expected.not_to contain_keystone_user_role('aodh@services') } - it { is_expected.to contain_keystone_service('aodh').with( + it { is_expected.to contain_keystone_service('aodh::alarming').with( :ensure => 'present', - :type => 'alarming', :description => 'OpenStack Alarming Service' ) } diff --git a/ceilometer/spec/classes/ceilometer_keystone_auth_spec.rb b/ceilometer/spec/classes/ceilometer_keystone_auth_spec.rb index 649b55a24..8701219cc 100644 --- a/ceilometer/spec/classes/ceilometer_keystone_auth_spec.rb +++ b/ceilometer/spec/classes/ceilometer_keystone_auth_spec.rb @@ -43,15 +43,14 @@ end it 'configures ceilometer service' do - is_expected.to contain_keystone_service( default_params[:auth_name] ).with( + is_expected.to contain_keystone_service("#{default_params[:auth_name]}::#{default_params[:service_type]}").with( :ensure => 'present', - :type => default_params[:service_type], :description => 'Openstack Metering Service' ) end it 'configure ceilometer endpoints' do - is_expected.to contain_keystone_endpoint("#{default_params[:region]}/#{default_params[:auth_name]}").with( + is_expected.to contain_keystone_endpoint("#{default_params[:region]}/#{default_params[:auth_name]}::#{default_params[:service_type]}").with( :ensure => 'present', :public_url => default_params[:public_url], :admin_url => default_params[:admin_url], @@ -90,15 +89,14 @@ end it 'configures ceilometer service' do - is_expected.to contain_keystone_service( params[:auth_name] ).with( + is_expected.to contain_keystone_service("#{params[:auth_name]}::#{params[:service_type]}").with( :ensure => 'present', - :type => params[:service_type], :description => 'Openstack Metering Service' ) end it 'configure ceilometer endpoints' do - is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}").with( + is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}::#{params[:service_type]}").with( :ensure => 'present', :public_url => params[:public_url], :admin_url => params[:admin_url], @@ -110,7 +108,7 @@ before do params.delete!(:configure_endpoint) it 'does not configure ceilometer endpoints' do - is_expected.to_not contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}") + is_expected.to_not contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}::#{params[:service_type]}") end end end @@ -127,12 +125,13 @@ :port => '65001', :public_protocol => 'https', :admin_protocol => 'ftp', - :internal_protocol => 'gopher' + :internal_protocol => 'gopher', + :service_type => 'metering', }) end it 'configure ceilometer endpoints' do - is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}").with( + is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}::#{params[:service_type]}").with( :ensure => 'present', :public_url => "#{params[:public_protocol]}://#{params[:public_address]}:#{params[:port]}", :admin_url => "#{params[:admin_protocol]}://#{params[:admin_address]}:#{params[:port]}", @@ -154,10 +153,10 @@ is_expected.to contain_keystone_user_role('ceilometer@services') end it 'configures correct service name' do - is_expected.to contain_keystone_service('ceilometer_service') + is_expected.to contain_keystone_service('ceilometer_service::metering') end it 'configures correct endpoint name' do - is_expected.to contain_keystone_endpoint('RegionOne/ceilometer_service') + is_expected.to contain_keystone_endpoint('RegionOne/ceilometer_service::metering') end end @@ -169,9 +168,8 @@ it { is_expected.to_not contain_keystone_user('ceilometer') } it { is_expected.to contain_keystone_user_role('ceilometer@services') } - it { is_expected.to contain_keystone_service('ceilometer').with( - :ensure => 'present', - :type => 'metering', + it { is_expected.to contain_keystone_service('ceilometer::metering').with( + :ensure => 'present', :description => 'Openstack Metering Service' )} end @@ -187,9 +185,8 @@ it { is_expected.to_not contain_keystone_user('ceilometer') } it { is_expected.to_not contain_keystone_user_role('ceilometer@services') } - it { is_expected.to contain_keystone_service('ceilometer').with( - :ensure => 'present', - :type => 'metering', + it { is_expected.to contain_keystone_service('ceilometer::metering').with( + :ensure => 'present', :description => 'Openstack Metering Service' )} end diff --git a/cinder/manifests/backend/rbd.pp b/cinder/manifests/backend/rbd.pp index 75f673933..f9d9f8a41 100644 --- a/cinder/manifests/backend/rbd.pp +++ b/cinder/manifests/backend/rbd.pp @@ -106,11 +106,10 @@ # Creates an empty file if it doesn't yet exist ensure_resource('file', $::cinder::params::ceph_init_override, {'ensure' => 'present'}) - ensure_resource('file_line', 'set initscript env', { + file_line { "set initscript env ${name}": line => $override_line, path => $::cinder::params::ceph_init_override, - match => $override_match, - notify => Service['cinder-volume'] - }) + notify => Service['cinder-volume'], + } } diff --git a/cinder/manifests/db.pp b/cinder/manifests/db.pp index 1e2e9231b..0c35d6b40 100644 --- a/cinder/manifests/db.pp +++ b/cinder/manifests/db.pp @@ -56,13 +56,17 @@ $database_max_overflow_real = pick($::cinder::database_max_overflow,$database_max_overflow) validate_re($database_connection_real, - '(sqlite|mysql|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') + '^(sqlite|mysql(\+pymysql)?|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') case $database_connection_real { - /^mysql:\/\//: { - $backend_package = false + /^mysql(\+pymysql)?:\/\//: { require 'mysql::bindings' require 'mysql::bindings::python' + if $database_connection_real =~ /^mysql\+pymysql/ { + $backend_package = $::cinder::params::pymysql_package_name + } else { + $backend_package = false + } } /^postgresql:\/\//: { $backend_package = false diff --git a/cinder/manifests/params.pp b/cinder/manifests/params.pp index adfa87858..8d2c6db09 100644 --- a/cinder/manifests/params.pp +++ b/cinder/manifests/params.pp @@ -21,6 +21,7 @@ $lio_package_name = 'targetcli' $lock_path = '/var/lock/cinder' $sqlite_package_name = 'python-pysqlite2' + $pymysql_package_name = 'python-pymysql' } elsif($::osfamily == 'RedHat') { @@ -41,6 +42,7 @@ $lio_package_name = 'targetcli' $lock_path = '/var/lib/cinder/tmp' $sqlite_package_name = undef + $pymysql_package_name = undef case $::operatingsystem { 'RedHat', 'CentOS', 'Scientific', 'OracleLinux': { diff --git a/cinder/spec/acceptance/basic_cinder_spec.rb b/cinder/spec/acceptance/basic_cinder_spec.rb index 597e1f4fb..eeaf6ebde 100644 --- a/cinder/spec/acceptance/basic_cinder_spec.rb +++ b/cinder/spec/acceptance/basic_cinder_spec.rb @@ -29,7 +29,7 @@ # Cinder resources class { '::cinder': - database_connection => 'mysql://cinder:a_big_secret@127.0.0.1/cinder?charset=utf8', + database_connection => 'mysql+pymysql://cinder:a_big_secret@127.0.0.1/cinder?charset=utf8', rabbit_userid => 'cinder', rabbit_password => 'an_even_bigger_secret', rabbit_host => '127.0.0.1', diff --git a/cinder/spec/classes/cinder_db_spec.rb b/cinder/spec/classes/cinder_db_spec.rb index 5e1fa09cb..bbf53ec02 100644 --- a/cinder/spec/classes/cinder_db_spec.rb +++ b/cinder/spec/classes/cinder_db_spec.rb @@ -16,19 +16,18 @@ context 'with specific parameters' do let :params do - { :database_connection => 'mysql://cinder:cinder@localhost/cinder', + { :database_connection => 'mysql+pymysql://cinder:cinder@localhost/cinder', :database_idle_timeout => '3601', :database_min_pool_size => '2', :database_max_retries => '11', :database_retry_interval => '11', } end - it { is_expected.to contain_cinder_config('database/connection').with_value('mysql://cinder:cinder@localhost/cinder').with_secret(true) } + it { is_expected.to contain_cinder_config('database/connection').with_value('mysql+pymysql://cinder:cinder@localhost/cinder').with_secret(true) } it { is_expected.to contain_cinder_config('database/idle_timeout').with_value('3601') } it { is_expected.to contain_cinder_config('database/min_pool_size').with_value('2') } it { is_expected.to contain_cinder_config('database/max_retries').with_value('11') } it { is_expected.to contain_cinder_config('database/retry_interval').with_value('11') } - end context 'with postgresql backend' do @@ -42,6 +41,14 @@ end + context 'with MySQL-python library as backend package' do + let :params do + { :database_connection => 'mysql://cinder:cinder@localhost/cinder', } + end + + it { is_expected.to contain_package('python-mysqldb').with(:ensure => 'present') } + end + context 'with incorrect database_connection string' do let :params do { :database_connection => 'redis://cinder:cinder@localhost/cinder', } @@ -50,6 +57,14 @@ it_raises 'a Puppet::Error', /validate_re/ end + context 'with incorrect pymysql database_connection string' do + let :params do + { :database_connection => 'foo+pymysql://cinder:cinder@localhost/cinder', } + end + + it_raises 'a Puppet::Error', /validate_re/ + end + end context 'on Debian platforms' do @@ -62,6 +77,20 @@ end it_configures 'cinder::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://cinder:cinder@localhost/cinder', } + end + + it 'install the proper backend package' do + is_expected.to contain_package('cinder-backend-package').with( + :ensure => 'present', + :name => 'python-pymysql', + :tag => 'openstack' + ) + end + end end context 'on Redhat platforms' do @@ -73,6 +102,14 @@ end it_configures 'cinder::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://cinder:cinder@localhost/cinder', } + end + + it { is_expected.not_to contain_package('cinder-backend-package') } + end end end diff --git a/cinder/spec/classes/cinder_keystone_auth_spec.rb b/cinder/spec/classes/cinder_keystone_auth_spec.rb index a07b523ba..38226708f 100644 --- a/cinder/spec/classes/cinder_keystone_auth_spec.rb +++ b/cinder/spec/classes/cinder_keystone_auth_spec.rb @@ -19,28 +19,26 @@ :ensure => 'present', :roles => ['admin'] ) - is_expected.to contain_keystone_service('cinder').with( + is_expected.to contain_keystone_service('cinder::volume').with( :ensure => 'present', - :type => 'volume', :description => 'Cinder Service' ) - is_expected.to contain_keystone_service('cinderv2').with( + is_expected.to contain_keystone_service('cinderv2::volumev2').with( :ensure => 'present', - :type => 'volumev2', :description => 'Cinder Service v2' ) end it 'configures keystone endpoints' do - is_expected.to contain_keystone_endpoint('RegionOne/cinder').with( + is_expected.to contain_keystone_endpoint('RegionOne/cinder::volume').with( :ensure => 'present', :public_url => 'http://127.0.0.1:8776/v1/%(tenant_id)s', :admin_url => 'http://127.0.0.1:8776/v1/%(tenant_id)s', :internal_url => 'http://127.0.0.1:8776/v1/%(tenant_id)s' ) - is_expected.to contain_keystone_endpoint('RegionOne/cinderv2').with( + is_expected.to contain_keystone_endpoint('RegionOne/cinderv2::volumev2').with( :ensure => 'present', :public_url => 'http://127.0.0.1:8776/v2/%(tenant_id)s', :admin_url => 'http://127.0.0.1:8776/v2/%(tenant_id)s', @@ -63,14 +61,14 @@ end it 'configures keystone endpoints' do - is_expected.to contain_keystone_endpoint('RegionThree/cinder').with( + is_expected.to contain_keystone_endpoint('RegionThree/cinder::volume').with( :ensure => 'present', :public_url => 'https://10.0.42.1:4242/v41/%(tenant_id)s', :admin_url => 'https://10.0.42.2:4242/v41/%(tenant_id)s', :internal_url => 'https://10.0.42.3:4242/v41/%(tenant_id)s' ) - is_expected.to contain_keystone_endpoint('RegionThree/cinderv2').with( + is_expected.to contain_keystone_endpoint('RegionThree/cinderv2::volumev2').with( :ensure => 'present', :public_url => 'https://10.0.42.1:4242/v42/%(tenant_id)s', :admin_url => 'https://10.0.42.2:4242/v42/%(tenant_id)s', @@ -95,14 +93,14 @@ end it 'configures keystone endpoints' do - is_expected.to contain_keystone_endpoint('RegionThree/cinder').with( + is_expected.to contain_keystone_endpoint('RegionThree/cinder::volume').with( :ensure => 'present', :public_url => 'https://10.0.42.1:4242/v42/%(tenant_id)s', :admin_url => 'https://10.0.42.2:4242/v42/%(tenant_id)s', :internal_url => 'https://10.0.42.3:4242/v42/%(tenant_id)s' ) - is_expected.to contain_keystone_endpoint('RegionThree/cinderv2').with( + is_expected.to contain_keystone_endpoint('RegionThree/cinderv2::volumev2').with( :ensure => 'present', :public_url => 'https://10.0.42.1:4242/v2/%(tenant_id)s', :admin_url => 'https://10.0.42.2:4242/v2/%(tenant_id)s', @@ -119,8 +117,8 @@ :configure_endpoint_v2 => false ) end - it { is_expected.to_not contain_keystone_endpoint('RegionOne/cinder') } - it { is_expected.to_not contain_keystone_endpoint('RegionOne/cinderv2') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/cinder::volume') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/cinderv2::volumev2') } end describe 'when user is_expected.to not be configured' do @@ -132,9 +130,8 @@ it { is_expected.to_not contain_keystone_user('cinder') } it { is_expected.to contain_keystone_user_role('cinder@services') } - it { is_expected.to contain_keystone_service('cinder').with( + it { is_expected.to contain_keystone_service('cinder::volume').with( :ensure => 'present', - :type => 'volume', :description => 'Cinder Service' ) } @@ -150,9 +147,8 @@ it { is_expected.to_not contain_keystone_user('cinder') } it { is_expected.to_not contain_keystone_user_role('cinder@services') } - it { is_expected.to contain_keystone_service('cinder').with( + it { is_expected.to contain_keystone_service('cinder::volume').with( :ensure => 'present', - :type => 'volume', :description => 'Cinder Service' ) } @@ -186,10 +182,10 @@ it { is_expected.to contain_keystone_user('cinder') } it { is_expected.to contain_keystone_user_role('cinder@services') } - it { is_expected.to contain_keystone_service('cinder_service') } - it { is_expected.to contain_keystone_service('cinder_service_v2') } - it { is_expected.to contain_keystone_endpoint('RegionOne/cinder_service') } - it { is_expected.to contain_keystone_endpoint('RegionOne/cinder_service_v2') } + it { is_expected.to contain_keystone_service('cinder_service::volume') } + it { is_expected.to contain_keystone_service('cinder_service_v2::volumev2') } + it { is_expected.to contain_keystone_endpoint('RegionOne/cinder_service::volume') } + it { is_expected.to contain_keystone_endpoint('RegionOne/cinder_service_v2::volumev2') } end diff --git a/cinder/spec/classes/cinder_volume_rbd_spec.rb b/cinder/spec/classes/cinder_volume_rbd_spec.rb index fc06c01f0..d19e84911 100644 --- a/cinder/spec/classes/cinder_volume_rbd_spec.rb +++ b/cinder/spec/classes/cinder_volume_rbd_spec.rb @@ -1,6 +1,7 @@ require 'spec_helper' describe 'cinder::volume::rbd' do + let :req_params do { :rbd_pool => 'volumes', @@ -35,7 +36,7 @@ is_expected.to contain_cinder_config('DEFAULT/rbd_user').with_value(req_params[:rbd_user]) is_expected.to contain_cinder_config('DEFAULT/rbd_secret_uuid').with_value(req_params[:rbd_secret_uuid]) is_expected.to contain_file('/etc/init/cinder-volume.override').with(:ensure => 'present') - is_expected.to contain_file_line('set initscript env').with( + is_expected.to contain_file_line('set initscript env DEFAULT').with( :line => /env CEPH_ARGS=\"--id test\"/, :path => '/etc/init/cinder-volume.override', :notify => 'Service[cinder-volume]') @@ -70,7 +71,7 @@ end it 'should configure RedHat init override' do - is_expected.to contain_file_line('set initscript env').with( + is_expected.to contain_file_line('set initscript env DEFAULT').with( :line => /export CEPH_ARGS=\"--id test\"/, :path => '/etc/sysconfig/openstack-cinder-volume', :notify => 'Service[cinder-volume]') diff --git a/cinder/spec/defines/cinder_backend_rbd_spec.rb b/cinder/spec/defines/cinder_backend_rbd_spec.rb index 02a26c0fa..f89fd9deb 100644 --- a/cinder/spec/defines/cinder_backend_rbd_spec.rb +++ b/cinder/spec/defines/cinder_backend_rbd_spec.rb @@ -44,7 +44,7 @@ is_expected.to contain_cinder_config("#{req_params[:volume_backend_name]}/rbd_secret_uuid").with_value(req_params[:rbd_secret_uuid]) is_expected.to contain_cinder_config("#{req_params[:volume_backend_name]}/backend_host").with_value('rbd:'"#{req_params[:rbd_pool]}") is_expected.to contain_file('/etc/init/cinder-volume.override').with(:ensure => 'present') - is_expected.to contain_file_line('set initscript env').with( + is_expected.to contain_file_line('set initscript env rbd-ssd').with( :line => /env CEPH_ARGS=\"--id test\"/, :path => '/etc/init/cinder-volume.override', :notify => 'Service[cinder-volume]') @@ -54,7 +54,7 @@ let :pre_condition do "cinder::backend::rbd { 'ceph2': rbd_pool => 'volumes2', - rbd_user => 'test' + rbd_user => 'test2' }" end it { is_expected.to contain_cinder_config("#{req_params[:volume_backend_name]}/volume_driver").with_value('cinder.volume.drivers.rbd.RBDDriver') } @@ -62,7 +62,7 @@ it { is_expected.to contain_cinder_config("#{req_params[:volume_backend_name]}/rbd_user").with_value(req_params[:rbd_user]) } it { is_expected.to contain_cinder_config("ceph2/volume_driver").with_value('cinder.volume.drivers.rbd.RBDDriver') } it { is_expected.to contain_cinder_config("ceph2/rbd_pool").with_value('volumes2') } - it { is_expected.to contain_cinder_config("ceph2/rbd_user").with_value('test') } + it { is_expected.to contain_cinder_config("ceph2/rbd_user").with_value('test2') } end context 'rbd backend with additional configuration' do @@ -107,7 +107,7 @@ end it 'should configure RedHat init override' do - is_expected.to contain_file_line('set initscript env').with( + is_expected.to contain_file_line('set initscript env rbd-ssd').with( :line => /export CEPH_ARGS=\"--id test\"/, :path => '/etc/sysconfig/openstack-cinder-volume', :notify => 'Service[cinder-volume]') diff --git a/glance/manifests/keystone/auth.pp b/glance/manifests/keystone/auth.pp index 52028c841..5fe551887 100644 --- a/glance/manifests/keystone/auth.pp +++ b/glance/manifests/keystone/auth.pp @@ -182,8 +182,8 @@ $real_service_name = pick($service_name, $auth_name) if $configure_endpoint { - Keystone_endpoint["${region}/${real_service_name}"] ~> Service <| name == 'glance-api' |> - Keystone_endpoint["${region}/${real_service_name}"] -> Glance_image<||> + Keystone_endpoint["${region}/${real_service_name}::${service_type}"] ~> Service<| title == 'glance-api' |> + Keystone_endpoint["${region}/${real_service_name}::${service_type}"] -> Glance_image<||> } keystone::resource::service_identity { $auth_name: @@ -203,8 +203,8 @@ } if $configure_user_role { - Keystone_user_role["${auth_name}@${tenant}"] ~> Service <| name == 'glance-registry' |> - Keystone_user_role["${auth_name}@${tenant}"] ~> Service <| name == 'glance-api' |> + Keystone_user_role["${auth_name}@${tenant}"] ~> Service<| title == 'glance-registry' |> + Keystone_user_role["${auth_name}@${tenant}"] ~> Service<| title == 'glance-api' |> } } diff --git a/glance/spec/acceptance/basic_glance_spec.rb b/glance/spec/acceptance/basic_glance_spec.rb index 4348a527c..b79bf77d6 100644 --- a/glance/spec/acceptance/basic_glance_spec.rb +++ b/glance/spec/acceptance/basic_glance_spec.rb @@ -3,9 +3,7 @@ describe 'glance class' do context 'default parameters' do - - it 'should work with no errors' do - pp= <<-EOS + pp= <<-EOS include ::openstack_integration include ::openstack_integration::repos include ::openstack_integration::mysql @@ -41,8 +39,29 @@ class { '::glance::registry': is_public => 'yes', source => 'http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-disk.img', } - EOS + EOS + + it 'should configure the glance endpoint before the glance-api service uses it' do + pp2 = pp + "Service['glance-api'] -> Keystone_endpoint['RegionOne/glance::image']" + expect(apply_manifest(pp2, :expect_failures => true, :noop => true).stderr).to match(/Found 1 dependency cycle/i) + end + + it 'should configure the glance user before the glance-api service uses it' do + pp2 = pp + "Service['glance-api'] -> Keystone_user_role['glance@services']" + expect(apply_manifest(pp2, :expect_failures => true, :noop => true).stderr).to match(/Found 1 dependency cycle/i) + end + it 'should configure the glance user before the glance-registry service uses it' do + pp2 = pp + "Service['glance-registry'] -> Keystone_user_role['glance@services']" + expect(apply_manifest(pp2, :expect_failures => true, :noop => true).stderr).to match(/Found 1 dependency cycle/i) + end + + it 'should configure the glance-api service before using it to provision glance_images' do + pp2 = pp + "Glance_image['test_image'] -> Service['glance-api']" + expect(apply_manifest(pp2, :expect_failures => true, :noop => true).stderr).to match(/Found 1 dependency cycle/i) + end + + it 'should work with no errors' do # Run it twice and test for idempotency apply_manifest(pp, :catch_failures => true) apply_manifest(pp, :catch_changes => true) diff --git a/glance/spec/classes/glance_keystone_auth_spec.rb b/glance/spec/classes/glance_keystone_auth_spec.rb index 55bfb5e91..09a881ece 100644 --- a/glance/spec/classes/glance_keystone_auth_spec.rb +++ b/glance/spec/classes/glance_keystone_auth_spec.rb @@ -18,13 +18,12 @@ :roles => ['admin'] ) } - it { is_expected.to contain_keystone_service('glance').with( + it { is_expected.to contain_keystone_service('glance::image').with( :ensure => 'present', - :type => 'image', :description => 'OpenStack Image Service' ) } - it { is_expected.to contain_keystone_endpoint('RegionOne/glance').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/glance::image').with( :ensure => 'present', :public_url => 'http://127.0.0.1:9292', :admin_url => 'http://127.0.0.1:9292', @@ -53,9 +52,8 @@ :roles => ['admin'] ) } - it { is_expected.to contain_keystone_service('glancey').with( + it { is_expected.to contain_keystone_service('glancey::imagey').with( :ensure => 'present', - :type => 'imagey', :description => 'OpenStack Image Service' ) } @@ -70,7 +68,7 @@ :admin_url => 'https://10.10.10.12:81/v2' } end - it { is_expected.to contain_keystone_endpoint('RegionTwo/glance').with( + it { is_expected.to contain_keystone_endpoint('RegionTwo/glance::image').with( :ensure => 'present', :public_url => 'https://10.10.10.10:81/v2', :internal_url => 'https://10.10.10.11:81/v2', @@ -94,7 +92,7 @@ } end - it { is_expected.to contain_keystone_endpoint('RegionTwo/glance').with( + it { is_expected.to contain_keystone_endpoint('RegionTwo/glance::image').with( :ensure => 'present', :public_url => 'https://10.0.0.1:9393', :admin_url => 'https://10.0.0.2:9393', @@ -112,7 +110,7 @@ } end - it { is_expected.to_not contain_keystone_endpoint('RegionOne/glance') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/glance::image') } end describe 'when disabling user configuration' do @@ -127,9 +125,8 @@ it { is_expected.to contain_keystone_user_role('glance@services') } - it { is_expected.to contain_keystone_service('glance').with( + it { is_expected.to contain_keystone_service('glance::image').with( :ensure => 'present', - :type => 'image', :description => 'OpenStack Image Service' ) } end @@ -147,9 +144,8 @@ it { is_expected.to_not contain_keystone_user_role('glance@services') } - it { is_expected.to contain_keystone_service('glance').with( + it { is_expected.to contain_keystone_service('glance::image').with( :ensure => 'present', - :type => 'image', :description => 'OpenStack Image Service' ) } end @@ -172,7 +168,7 @@ } end - it { is_expected.to contain_keystone_endpoint('RegionOne/glance').with_notify(["Service[glance-api]"]) } + it { is_expected.to contain_keystone_endpoint('RegionOne/glance::image').with_notify(["Service[glance-api]"]) } end describe 'when overriding service name' do @@ -186,8 +182,8 @@ it { is_expected.to contain_keystone_user('glance') } it { is_expected.to contain_keystone_user_role('glance@services') } - it { is_expected.to contain_keystone_service('glance_service') } - it { is_expected.to contain_keystone_endpoint('RegionOne/glance_service') } + it { is_expected.to contain_keystone_service('glance_service::image') } + it { is_expected.to contain_keystone_endpoint('RegionOne/glance_service::image') } end diff --git a/gnocchi/examples/site.pp b/gnocchi/examples/site.pp index 2598a40ea..d7e00f53b 100644 --- a/gnocchi/examples/site.pp +++ b/gnocchi/examples/site.pp @@ -25,3 +25,5 @@ archive_policy_name => 'high', flush_delay => '100', } + +include ::gnocchi::client diff --git a/gnocchi/manifests/client.pp b/gnocchi/manifests/client.pp new file mode 100644 index 000000000..f7f209006 --- /dev/null +++ b/gnocchi/manifests/client.pp @@ -0,0 +1,21 @@ +# +# Installs the gnocchi python library. +# +# == parameters +# [*ensure*] +# ensure state for package. +# +class gnocchi::client ( + $ensure = 'present' +) { + + include ::gnocchi::params + + package { 'python-gnocchiclient': + ensure => $ensure, + name => $::gnocchi::params::client_package_name, + tag => 'openstack', + } + +} + diff --git a/gnocchi/manifests/params.pp b/gnocchi/manifests/params.pp index d170b9cb5..0308dd3e2 100644 --- a/gnocchi/manifests/params.pp +++ b/gnocchi/manifests/params.pp @@ -12,6 +12,7 @@ $carbonara_package_name = 'openstack-gnocchi-carbonara' $statsd_package_name = 'openstack-gnocchi-statsd' $statsd_service_name = 'openstack-gnocchi-statsd' + $client_package_name = 'python-gnocchiclient' $gnocchi_wsgi_script_path = '/var/www/cgi-bin/gnocchi' $gnocchi_wsgi_script_source = '/usr/lib/python2.7/site-packages/gnocchi/rest/app.wsgi' } @@ -24,6 +25,7 @@ $carbonara_package_name = 'gnocchi-carbonara' $statsd_package_name = 'gnocchi-statsd' $statsd_service_name = 'gnocchi-statsd' + $client_package_name = 'python-gnocchiclient' $gnocchi_wsgi_script_path = '/usr/lib/cgi-bin/gnocchi' $gnocchi_wsgi_script_source = '/usr/share/gnocchi-common/app.wsgi' } diff --git a/gnocchi/manifests/statsd.pp b/gnocchi/manifests/statsd.pp index 6f27ee64f..afbc25733 100644 --- a/gnocchi/manifests/statsd.pp +++ b/gnocchi/manifests/statsd.pp @@ -2,6 +2,18 @@ # # == Parameters # +# [*resource_id*] +# (required) Resource UUID to use to identify statsd in Gnocchi. +# +# [*user_id*] +# (required) User UUID to use to identify statsd in Gnocchi. +# +# [*project_id*] +# (required) Project UUID to use to identify statsd in Gnocchi. +# +# [*flush_delay*] +# (required) Delay between flushes. +# # [*enabled*] # (optional) Should the service be enabled. # Defaults to true @@ -14,26 +26,10 @@ # (optional) Whether the service should be managed by Puppet. # Defaults to true. # -# [*resource_id*] -# (optional) Resource UUID to use to identify statsd in Gnocchi. -# Defaults to undef. -# -# [*user_id*] -# (optional) User UUID to use to identify statsd in Gnocchi. -# Defaults to undef. -# -# [*project_id*] -# (optional) Project UUID to use to identify statsd in Gnocchi. -# Defaults to undef. -# # [*archive_policy_name*] # (optional) Archive policy name to use when creating metrics. # Defaults to undef. # -# [*flush_delay*] -# (optional) Delay between flushes. -# Defaults to undef. -# class gnocchi::statsd ( $resource_id, $user_id, diff --git a/gnocchi/spec/acceptance/basic_gnocchi_spec.rb b/gnocchi/spec/acceptance/basic_gnocchi_spec.rb index 29abe659a..b93abe320 100644 --- a/gnocchi/spec/acceptance/basic_gnocchi_spec.rb +++ b/gnocchi/spec/acceptance/basic_gnocchi_spec.rb @@ -47,6 +47,7 @@ class { '::gnocchi::statsd': user_id => 'f81e9b1f-9505-4298-bc33-43dfbd9a973b', project_id => '203ef419-e73f-4b8a-a73f-3d599a72b18d', } + class { '::gnocchi::client': } } } EOS diff --git a/gnocchi/spec/classes/gnocchi_client_spec.rb b/gnocchi/spec/classes/gnocchi_client_spec.rb new file mode 100644 index 000000000..87151bc3e --- /dev/null +++ b/gnocchi/spec/classes/gnocchi_client_spec.rb @@ -0,0 +1,33 @@ +require 'spec_helper' + +describe 'gnocchi::client' do + + shared_examples_for 'gnocchi client' do + + it { is_expected.to contain_class('gnocchi::params') } + + it 'installs gnocchi client package' do + is_expected.to contain_package('python-gnocchiclient').with( + :ensure => 'present', + :name => 'python-gnocchiclient', + :tag => 'openstack', + ) + end + end + + context 'on Debian platforms' do + let :facts do + { :osfamily => 'Debian' } + end + + it_configures 'gnocchi client' + end + + context 'on RedHat platforms' do + let :facts do + { :osfamily => 'RedHat' } + end + + it_configures 'gnocchi client' + end +end diff --git a/gnocchi/spec/classes/gnocchi_keystone_auth_spec.rb b/gnocchi/spec/classes/gnocchi_keystone_auth_spec.rb index ae76fb968..e5d53aefe 100644 --- a/gnocchi/spec/classes/gnocchi_keystone_auth_spec.rb +++ b/gnocchi/spec/classes/gnocchi_keystone_auth_spec.rb @@ -26,13 +26,12 @@ :roles => ['admin'] )} - it { is_expected.to contain_keystone_service('gnocchi').with( + it { is_expected.to contain_keystone_service('gnocchi::metric').with( :ensure => 'present', - :type => 'metric', :description => 'OpenStack Metric Service' ) } - it { is_expected.to contain_keystone_endpoint('RegionOne/gnocchi').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/gnocchi::metric').with( :ensure => 'present', :public_url => 'http://127.0.0.1:8041', :admin_url => 'http://127.0.0.1:8041', @@ -48,7 +47,7 @@ :admin_url => 'http://10.10.10.12:81' } end - it { is_expected.to contain_keystone_endpoint('RegionOne/gnocchi').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/gnocchi::metric').with( :ensure => 'present', :public_url => 'https://10.10.10.10:80', :internal_url => 'http://10.10.10.11:81', @@ -64,8 +63,8 @@ it { is_expected.to contain_keystone_user('gnocchiany') } it { is_expected.to contain_keystone_user_role('gnocchiany@services') } - it { is_expected.to contain_keystone_service('gnocchiany') } - it { is_expected.to contain_keystone_endpoint('RegionOne/gnocchiany') } + it { is_expected.to contain_keystone_service('gnocchiany::metric') } + it { is_expected.to contain_keystone_endpoint('RegionOne/gnocchiany::metric') } end describe 'when overriding service name' do @@ -77,8 +76,8 @@ it { is_expected.to contain_keystone_user('gnocchi') } it { is_expected.to contain_keystone_user_role('gnocchi@services') } - it { is_expected.to contain_keystone_service('gnocchi_service') } - it { is_expected.to contain_keystone_endpoint('RegionOne/gnocchi_service') } + it { is_expected.to contain_keystone_service('gnocchi_service::metric') } + it { is_expected.to contain_keystone_endpoint('RegionOne/gnocchi_service::metric') } end describe 'when disabling user configuration' do @@ -92,9 +91,8 @@ it { is_expected.not_to contain_keystone_user('gnocchi') } it { is_expected.to contain_keystone_user_role('gnocchi@services') } - it { is_expected.to contain_keystone_service('gnocchi').with( + it { is_expected.to contain_keystone_service('gnocchi::metric').with( :ensure => 'present', - :type => 'metric', :description => 'OpenStack Metric Service' ) } @@ -112,9 +110,8 @@ it { is_expected.not_to contain_keystone_user('gnocchi') } it { is_expected.not_to contain_keystone_user_role('gnocchi@services') } - it { is_expected.to contain_keystone_service('gnocchi').with( + it { is_expected.to contain_keystone_service('gnocchi::metric').with( :ensure => 'present', - :type => 'metric', :description => 'OpenStack Metric Service' ) } diff --git a/heat/examples/site.pp b/heat/examples/site.pp index 0e2b79c4c..2a0f902cd 100644 --- a/heat/examples/site.pp +++ b/heat/examples/site.pp @@ -15,7 +15,7 @@ class { '::heat': # The keystone_password parameter is mandatory keystone_password => 'password', - sql_connection => 'mysql://heat:heat@localhost/heat' + sql_connection => 'mysql+pymysql://heat:heat@localhost/heat' } # Install heat-engine diff --git a/heat/manifests/db.pp b/heat/manifests/db.pp index 957aa4b11..17d6334c2 100644 --- a/heat/manifests/db.pp +++ b/heat/manifests/db.pp @@ -49,6 +49,7 @@ ) { include ::heat::deps + include ::heat::params # NOTE(spredzy): In order to keep backward compatibility we rely on the pick function # to use heat:: if heat::db:: isn't specified. @@ -62,14 +63,18 @@ $sync_db_real = pick($::heat::sync_db, $sync_db) validate_re($database_connection_real, - '(sqlite|mysql|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') + '^(sqlite|mysql(\+pymysql)?|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') if $database_connection_real { case $database_connection_real { - /^mysql:\/\//: { - $backend_package = false + /^mysql(\+pymysql)?:\/\//: { require 'mysql::bindings' require 'mysql::bindings::python' + if $database_connection_real =~ /^mysql\+pymysql/ { + $backend_package = $::heat::params::pymysql_package_name + } else { + $backend_package = false + } } /^postgresql:\/\//: { $backend_package = false diff --git a/heat/manifests/params.pp b/heat/manifests/params.pp index d8b4830e5..1133d68bb 100644 --- a/heat/manifests/params.pp +++ b/heat/manifests/params.pp @@ -17,6 +17,7 @@ $client_package_name = 'python-heatclient' $common_package_name = 'openstack-heat-common' $sqlite_package_name = undef + $pymysql_package_name = undef # service names $api_service_name = 'openstack-heat-api' $api_cloudwatch_service_name = 'openstack-heat-api-cloudwatch' @@ -32,6 +33,7 @@ $client_package_name = 'python-heatclient' $common_package_name = 'heat-common' $sqlite_package_name = 'python-pysqlite2' + $pymysql_package_name = 'python-pymysql' # service names $api_service_name = 'heat-api' $api_cloudwatch_service_name = 'heat-api-cloudwatch' diff --git a/heat/spec/acceptance/basic_heat_spec.rb b/heat/spec/acceptance/basic_heat_spec.rb index 81ca93dde..9021f2943 100644 --- a/heat/spec/acceptance/basic_heat_spec.rb +++ b/heat/spec/acceptance/basic_heat_spec.rb @@ -32,7 +32,7 @@ class { '::heat': rabbit_userid => 'heat', rabbit_password => 'an_even_bigger_secret', rabbit_host => '127.0.0.1', - database_connection => 'mysql://heat:a_big_secret@127.0.0.1/heat?charset=utf8', + database_connection => 'mysql+pymysql://heat:a_big_secret@127.0.0.1/heat?charset=utf8', identity_uri => 'http://127.0.0.1:35357/', keystone_password => 'a_big_secret', debug => true, diff --git a/heat/spec/classes/heat_db_spec.rb b/heat/spec/classes/heat_db_spec.rb index a138827a9..585556dff 100644 --- a/heat/spec/classes/heat_db_spec.rb +++ b/heat/spec/classes/heat_db_spec.rb @@ -17,7 +17,7 @@ context 'with specific parameters' do let :params do - { :database_connection => 'mysql://heat:heat@localhost/heat', + { :database_connection => 'mysql+pymysql://heat:heat@localhost/heat', :database_idle_timeout => '3601', :database_min_pool_size => '2', :database_max_retries => '11', @@ -26,7 +26,7 @@ end it { is_expected.not_to contain_class('heat::db::sync') } - it { is_expected.to contain_heat_config('database/connection').with_value('mysql://heat:heat@localhost/heat').with_secret(true) } + it { is_expected.to contain_heat_config('database/connection').with_value('mysql+pymysql://heat:heat@localhost/heat').with_secret(true) } it { is_expected.to contain_heat_config('database/idle_timeout').with_value('3601') } it { is_expected.to contain_heat_config('database/min_pool_size').with_value('2') } it { is_expected.to contain_heat_config('database/max_retries').with_value('11') } @@ -34,6 +34,14 @@ end + context 'with MySQL-python library as backend package' do + let :params do + { :database_connection => 'mysql://heat:heat@localhost/heat' } + end + + it { is_expected.to contain_heat_config('database/connection').with_value('mysql://heat:heat@localhost/heat').with_secret(true) } + end + context 'with postgresql backend' do let :params do { :database_connection => 'postgresql://heat:heat@localhost/heat', } @@ -53,6 +61,14 @@ it_raises 'a Puppet::Error', /validate_re/ end + context 'with incorrect database_connection string' do + let :params do + { :database_connection => 'foo+pymysql://heat:heat@localhost/heat', } + end + + it_raises 'a Puppet::Error', /validate_re/ + end + end context 'on Debian platforms' do @@ -65,6 +81,14 @@ end it_configures 'heat::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://heat:heat@localhost/heat' } + end + + it { is_expected.to contain_package('heat-backend-package').with({ :ensure => 'present', :name => 'python-pymysql' }) } + end end context 'on Redhat platforms' do @@ -76,6 +100,14 @@ end it_configures 'heat::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://heat:heat@localhost/heat' } + end + + it { is_expected.not_to contain_package('heat-backend-package') } + end end end diff --git a/heat/spec/classes/heat_init_spec.rb b/heat/spec/classes/heat_init_spec.rb index 74bc6a90a..8194a8009 100644 --- a/heat/spec/classes/heat_init_spec.rb +++ b/heat/spec/classes/heat_init_spec.rb @@ -14,7 +14,7 @@ :rabbit_userid => 'guest', :rabbit_password => '', :rabbit_virtual_host => '/', - :database_connection => 'mysql://user@host/database', + :database_connection => 'mysql+pymysql://user@host/database', :database_idle_timeout => 3600, :auth_uri => 'http://127.0.0.1:5000/v2.0', :keystone_ec2_uri => 'http://127.0.0.1:5000/v2.0/ec2tokens', diff --git a/heat/spec/classes/heat_keystone_auth_cfn_spec.rb b/heat/spec/classes/heat_keystone_auth_cfn_spec.rb index 0f9bba321..80af096eb 100644 --- a/heat/spec/classes/heat_keystone_auth_cfn_spec.rb +++ b/heat/spec/classes/heat_keystone_auth_cfn_spec.rb @@ -45,15 +45,14 @@ end it 'configures heat service' do - is_expected.to contain_keystone_service( params[:auth_name] ).with( + is_expected.to contain_keystone_service("#{params[:auth_name]}::#{params[:service_type]}").with( :ensure => 'present', - :type => params[:service_type], :description => 'Openstack Cloudformation Service' ) end it 'configure heat endpoints' do - is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}").with( + is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}::#{params[:service_type]}").with( :ensure => 'present', :public_url => params[:public_url], :admin_url => params[:admin_url], @@ -67,7 +66,7 @@ :configure_service => false }) end - it { is_expected.to_not contain_keystone_service("#{params[:region]}/#{params[:auth_name]}") } + it { is_expected.to_not contain_keystone_service("#{params[:auth_name]}::#{params[:service_type]}") } end end @@ -85,7 +84,7 @@ }) end - it { is_expected.to contain_keystone_endpoint('RegionOne/heat-cfn').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/heat-cfn::cloudformation').with( :ensure => 'present', :public_url => "#{params[:public_protocol]}://#{params[:public_address]}:#{params[:port]}/#{params[:version]}", :admin_url => "#{params[:admin_protocol]}://#{params[:admin_address]}:#{params[:port]}/#{params[:version]}", @@ -106,10 +105,10 @@ is_expected.to contain_keystone_user_role('heat-cfn@services') end it 'configures correct service name' do - is_expected.to contain_keystone_service('heat-cfn_service') + is_expected.to contain_keystone_service('heat-cfn_service::cloudformation') end it 'configures correct endpoint name' do - is_expected.to contain_keystone_endpoint('RegionOne/heat-cfn_service') + is_expected.to contain_keystone_endpoint('RegionOne/heat-cfn_service::cloudformation') end end @@ -121,9 +120,8 @@ it { is_expected.to_not contain_keystone_user('heat_cfn') } it { is_expected.to contain_keystone_user_role('heat-cfn@services') } - it { is_expected.to contain_keystone_service('heat-cfn').with( + it { is_expected.to contain_keystone_service('heat-cfn::cloudformation').with( :ensure => 'present', - :type => 'cloudformation', :description => 'Openstack Cloudformation Service' )} end @@ -139,9 +137,8 @@ it { is_expected.to_not contain_keystone_user('heat_cfn') } it { is_expected.to_not contain_keystone_user_role('heat-cfn@services') } - it { is_expected.to contain_keystone_service('heat-cfn').with( + it { is_expected.to contain_keystone_service('heat-cfn::cloudformation').with( :ensure => 'present', - :type => 'cloudformation', :description => 'Openstack Cloudformation Service' )} end diff --git a/heat/spec/classes/heat_keystone_auth_spec.rb b/heat/spec/classes/heat_keystone_auth_spec.rb index 6d9e1a8e2..84e1a5f49 100644 --- a/heat/spec/classes/heat_keystone_auth_spec.rb +++ b/heat/spec/classes/heat_keystone_auth_spec.rb @@ -17,7 +17,7 @@ before do params.merge!({:configure_service => false}) end - it { is_expected.to_not contain_keystone_service('RegionOne/heat') } + it { is_expected.to_not contain_keystone_service('heat::orchestration') } end context 'with overridden parameters' do @@ -61,15 +61,14 @@ end it 'configures heat service' do - is_expected.to contain_keystone_service( params[:auth_name] ).with( + is_expected.to contain_keystone_service("#{params[:auth_name]}::#{params[:service_type]}").with( :ensure => 'present', - :type => params[:service_type], :description => 'Openstack Orchestration Service' ) end it 'configure heat endpoints' do - is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}").with( + is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}::#{params[:service_type]}").with( :ensure => 'present', :public_url => params[:public_url], :admin_url => params[:admin_url], @@ -92,7 +91,7 @@ }) end - it { is_expected.to contain_keystone_endpoint('RegionOne/heat').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/heat::orchestration').with( :ensure => 'present', :public_url => "#{params[:public_protocol]}://#{params[:public_address]}:#{params[:port]}/#{params[:version]}/%(tenant_id)s", :admin_url => "#{params[:admin_protocol]}://#{params[:admin_address]}:#{params[:port]}/#{params[:version]}/%(tenant_id)s", @@ -113,10 +112,10 @@ is_expected.to contain_keystone_user_role('heat@services') end it 'configures correct service name' do - is_expected.to contain_keystone_service('heat_service') + is_expected.to contain_keystone_service('heat_service::orchestration') end it 'configures correct endpoint name' do - is_expected.to contain_keystone_endpoint('RegionOne/heat_service') + is_expected.to contain_keystone_endpoint('RegionOne/heat_service::orchestration') end end @@ -128,9 +127,8 @@ it { is_expected.to_not contain_keystone_user('heat') } it { is_expected.to contain_keystone_user_role('heat@services') } - it { is_expected.to contain_keystone_service('heat').with( + it { is_expected.to contain_keystone_service('heat::orchestration').with( :ensure => 'present', - :type => 'orchestration', :description => 'Openstack Orchestration Service' )} end @@ -146,9 +144,8 @@ it { is_expected.to_not contain_keystone_user('heat') } it { is_expected.to_not contain_keystone_user_role('heat@services') } - it { is_expected.to contain_keystone_service('heat').with( + it { is_expected.to contain_keystone_service('heat::orchestration').with( :ensure => 'present', - :type => 'orchestration', :description => 'Openstack Orchestration Service' )} end diff --git a/ironic/manifests/init.pp b/ironic/manifests/init.pp index aa5442c0a..0bd03662c 100644 --- a/ironic/manifests/init.pp +++ b/ironic/manifests/init.pp @@ -284,7 +284,6 @@ ensure => $package_ensure, name => $::ironic::params::common_package_name, tag => ['openstack', 'ironic-package'], - notify => Exec['ironic-dbsync'], } validate_array($enabled_drivers) diff --git a/ironic/manifests/keystone/auth.pp b/ironic/manifests/keystone/auth.pp index b6519fa3b..63d944fa0 100644 --- a/ironic/manifests/keystone/auth.pp +++ b/ironic/manifests/keystone/auth.pp @@ -193,7 +193,7 @@ Keystone_user_role["${auth_name}@${tenant}"] ~> Service <| name == 'ironic-server' |> } - Keystone_endpoint["${region}/${real_service_name}"] ~> Service <| name == 'ironic-server' |> + Keystone_endpoint["${region}/${real_service_name}::${service_type}"] ~> Service <| name == 'ironic-server' |> keystone::resource::service_identity { $auth_name: configure_user => $configure_user, diff --git a/ironic/spec/classes/ironic_keystone_auth_spec.rb b/ironic/spec/classes/ironic_keystone_auth_spec.rb index 83e62207f..448fa5f06 100644 --- a/ironic/spec/classes/ironic_keystone_auth_spec.rb +++ b/ironic/spec/classes/ironic_keystone_auth_spec.rb @@ -42,13 +42,12 @@ :roles => ['admin'] )} - it { is_expected.to contain_keystone_service('ironic').with( + it { is_expected.to contain_keystone_service('ironic::baremetal').with( :ensure => 'present', - :type => 'baremetal', :description => 'Ironic Bare Metal Provisioning Service' ) } - it { is_expected.to contain_keystone_endpoint('RegionOne/ironic').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/ironic::baremetal').with( :ensure => 'present', :public_url => "http://127.0.0.1:6385", :admin_url => "http://127.0.0.1:6385", @@ -77,7 +76,7 @@ :internal_url => 'https://10.0.0.11:6385' } end - it { is_expected.to contain_keystone_endpoint('RegionOne/ironic').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/ironic::baremetal').with( :ensure => 'present', :public_url => 'https://10.0.0.10:6385', :admin_url => 'https://10.0.0.11:6385', @@ -96,7 +95,7 @@ :admin_address => '10.10.10.12' } end - it { is_expected.to contain_keystone_endpoint('RegionOne/ironic').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/ironic::baremetal').with( :ensure => 'present', :public_url => "https://10.10.10.10:80", :internal_url => "http://10.10.10.11:81", @@ -112,8 +111,8 @@ it { is_expected.to contain_keystone_user('ironicy') } it { is_expected.to contain_keystone_user_role('ironicy@services') } - it { is_expected.to contain_keystone_service('ironicy') } - it { is_expected.to contain_keystone_endpoint('RegionOne/ironicy') } + it { is_expected.to contain_keystone_service('ironicy::baremetal') } + it { is_expected.to contain_keystone_endpoint('RegionOne/ironicy::baremetal') } end describe 'when overriding service name' do @@ -126,8 +125,8 @@ it { is_expected.to contain_keystone_user('ironic') } it { is_expected.to contain_keystone_user_role('ironic@services') } - it { is_expected.to contain_keystone_service('ironic_service') } - it { is_expected.to contain_keystone_endpoint('RegionOne/ironic_service') } + it { is_expected.to contain_keystone_service('ironic_service::baremetal') } + it { is_expected.to contain_keystone_endpoint('RegionOne/ironic_service::baremetal') } end describe 'when disabling user configuration' do @@ -143,9 +142,8 @@ it { is_expected.to contain_keystone_user_role('ironic@services') } - it { is_expected.to contain_keystone_service('ironic').with( + it { is_expected.to contain_keystone_service('ironic::baremetal').with( :ensure => 'present', - :type => 'baremetal', :description => 'Ironic Bare Metal Provisioning Service' ) } @@ -165,9 +163,8 @@ it { is_expected.not_to contain_keystone_user_role('ironic@services') } - it { is_expected.to contain_keystone_service('ironic').with( + it { is_expected.to contain_keystone_service('ironic::baremetal').with( :ensure => 'present', - :type => 'baremetal', :description => 'Ironic Bare Metal Provisioning Service' ) } diff --git a/keystone/manifests/endpoint.pp b/keystone/manifests/endpoint.pp index c9e7ef5ea..a0891633f 100644 --- a/keystone/manifests/endpoint.pp +++ b/keystone/manifests/endpoint.pp @@ -36,12 +36,10 @@ # If keystone_project_domain is not specified, use $keystone_default_domain # Defaults to undef # -# === DEPRECATED -# # [*version*] # (optional) API version for endpoint. -# Defaults to 'v2.0' -# If the version is assigned to null value (forced to undef), then it won't be +# Defaults to 'v2.0'. Valid values are 'v2.0', 'v3', or the empty string ''. +# If the version is set to the empty string (''), then it won't be # used. This is the expected behaviour since Keystone V3 handles API versions # from the context. # @@ -61,10 +59,16 @@ $user_domain = undef, $project_domain = undef, $default_domain = undef, - $version = 'v2.0', # DEPRECATED + $version = 'unset', # defaults to 'v2.0' if unset by user ) { - if empty($version) { + if $version == 'unset' { + warning('In Mitaka, the default value of $keystone::endpoint::version will change to \'\'. To avoid this warning, please set the version parameter.') + $_version = 'v2.0' + } else { + $_version = $version + } + if empty($_version) { $admin_url_real = $admin_url $public_url_real = $public_url @@ -76,16 +80,14 @@ } } else { - warning('The version parameter is deprecated in Liberty.') - - $public_url_real = "${public_url}/${version}" - $admin_url_real = "${admin_url}/${version}" + $public_url_real = "${public_url}/${_version}" + $admin_url_real = "${admin_url}/${_version}" if $internal_url { - $internal_url_real = "${internal_url}/${version}" + $internal_url_real = "${internal_url}/${_version}" } else { - $internal_url_real = "${public_url}/${version}" + $internal_url_real = "${public_url}/${_version}" } } diff --git a/keystone/manifests/resource/service_identity.pp b/keystone/manifests/resource/service_identity.pp index 5ad3f9ce6..2298201b0 100644 --- a/keystone/manifests/resource/service_identity.pp +++ b/keystone/manifests/resource/service_identity.pp @@ -168,9 +168,8 @@ if $configure_service { if $service_type { - ensure_resource('keystone_service', $service_name_real, { + ensure_resource('keystone_service', "${service_name_real}::${service_type}", { 'ensure' => 'present', - 'type' => $service_type, 'description' => $service_description, }) } else { @@ -179,15 +178,29 @@ } if $configure_endpoint { - if $public_url and $admin_url and $internal_url { - ensure_resource('keystone_endpoint', "${region}/${service_name_real}", { - 'ensure' => 'present', - 'public_url' => $public_url, - 'admin_url' => $admin_url, - 'internal_url' => $internal_url, - }) + if $service_type { + if $public_url and $admin_url and $internal_url { + ensure_resource('keystone_endpoint', "${region}/${service_name_real}::${service_type}", { + 'ensure' => 'present', + 'public_url' => $public_url, + 'admin_url' => $admin_url, + 'internal_url' => $internal_url, + }) + } else { + fail ('When configuring an endpoint, you need to set the _url parameters.') + } } else { - fail ('When configuring an endpoint, you need to set the _url parameters.') + if $public_url and $admin_url and $internal_url { + ensure_resource('keystone_endpoint', "${region}/${service_name_real}", { + 'ensure' => 'present', + 'public_url' => $public_url, + 'admin_url' => $admin_url, + 'internal_url' => $internal_url, + }) + } else { + fail ('When configuring an endpoint, you need to set the _url parameters.') + } + warning('Defining a endpoint without the type is supported in Liberty and will be dropped in Mitaka. See https://bugs.launchpad.net/puppet-keystone/+bug/1506996') } } } diff --git a/keystone/spec/classes/keystone_endpoint_spec.rb b/keystone/spec/classes/keystone_endpoint_spec.rb index bafad575e..ec6f6751d 100644 --- a/keystone/spec/classes/keystone_endpoint_spec.rb +++ b/keystone/spec/classes/keystone_endpoint_spec.rb @@ -2,14 +2,13 @@ describe 'keystone::endpoint' do - it { is_expected.to contain_keystone_service('keystone').with( + it { is_expected.to contain_keystone_service('keystone::identity').with( :ensure => 'present', - :type => 'identity', :description => 'OpenStack Identity Service' )} describe 'with default parameters' do - it { is_expected.to contain_keystone_endpoint('RegionOne/keystone').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/keystone::identity').with( :ensure => 'present', :public_url => 'http://127.0.0.1:5000/v2.0', :admin_url => 'http://127.0.0.1:35357/v2.0', @@ -26,7 +25,7 @@ :internal_url => 'https://identity-int.some.tld/some/internal/endpoint' } end - it { is_expected.to contain_keystone_endpoint('RegionOne/keystone').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/keystone::identity').with( :ensure => 'present', :public_url => 'https://identity.some.tld/the/main/endpoint/v42.6', :admin_url => 'https://identity-int.some.tld/some/admin/endpoint/v42.6', @@ -41,7 +40,7 @@ { :version => '' } end - it { is_expected.to contain_keystone_endpoint('RegionOne/keystone').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/keystone::identity').with( :ensure => 'present', :public_url => 'http://127.0.0.1:5000', :admin_url => 'http://127.0.0.1:35357', @@ -56,7 +55,7 @@ end it 'internal_url should default to public_url' do - is_expected.to contain_keystone_endpoint('RegionOne/keystone').with( + is_expected.to contain_keystone_endpoint('RegionOne/keystone::identity').with( :ensure => 'present', :public_url => 'https://identity.some.tld/the/main/endpoint/v2.0', :internal_url => 'https://identity.some.tld/the/main/endpoint/v2.0' diff --git a/keystone/spec/defines/keystone_resource_service_identity_spec.rb b/keystone/spec/defines/keystone_resource_service_identity_spec.rb index ae4dad9c6..2e5834b4d 100644 --- a/keystone/spec/defines/keystone_resource_service_identity_spec.rb +++ b/keystone/spec/defines/keystone_resource_service_identity_spec.rb @@ -47,12 +47,26 @@ :roles => ['admin'], )} - it { is_expected.to contain_keystone_service(title).with( + it { is_expected.to contain_keystone_service("#{title}::network").with( :ensure => 'present', - :type => 'network', :description => 'neutron service', )} + it { is_expected.to contain_keystone_endpoint("RegionOne/#{title}::network").with( + :ensure => 'present', + :public_url => 'http://7.7.7.7:9696', + :internal_url => 'http://10.0.0.1:9696', + :admin_url => 'http://192.168.0.1:9696', + )} + end + + context 'when trying to create an endpoint without service_type (will be dropped in Mitaka)' do + let :params do + required_params.merge( + :configure_service => false, + :service_type => false, + ) + end it { is_expected.to contain_keystone_endpoint("RegionOne/#{title}").with( :ensure => 'present', :public_url => 'http://7.7.7.7:9696', diff --git a/manila/manifests/db.pp b/manila/manifests/db.pp index 823a98ac1..6b67612bd 100644 --- a/manila/manifests/db.pp +++ b/manila/manifests/db.pp @@ -43,6 +43,8 @@ $database_max_overflow = 20, ) { + include ::manila::params + # NOTE(spredzy): In order to keep backward compatibility we rely on the pick function # to use manila:: if manila::db:: isn't specified. $database_connection_real = pick($::manila::sql_connection, $database_connection) @@ -54,14 +56,18 @@ $database_max_overflow_real = pick($::manila::database_max_overflow, $database_max_overflow) validate_re($database_connection_real, - '(sqlite|mysql|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') + '^(sqlite|mysql(\+pymysql)?|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') if $database_connection_real { case $database_connection_real { - /^mysql:\/\//: { - $backend_package = false + /^mysql(\+pymysql)?:\/\//: { require 'mysql::bindings' require 'mysql::bindings::python' + if $database_connection_real =~ /^mysql\+pymysql/ { + $backend_package = $::manila::params::pymysql_package_name + } else { + $backend_package = false + } } /^postgresql:\/\//: { $backend_package = false diff --git a/manila/manifests/params.pp b/manila/manifests/params.pp index d92d83075..ef3c92b8f 100644 --- a/manila/manifests/params.pp +++ b/manila/manifests/params.pp @@ -22,6 +22,7 @@ $gluster_client_package_name = 'glusterfs-client' $gluster_package_name = 'glusterfs-common' $sqlite_package_name = 'python-pysqlite2' + $pymysql_package_name = 'python-pymysql' } elsif($::osfamily == 'RedHat') { @@ -41,6 +42,7 @@ $gluster_client_package_name = 'glusterfs-fuse' $gluster_package_name = 'glusterfs' $sqlite_package_name = undef + $pymysql_package_name = undef if $::operatingsystem == 'RedHat' and (versioncmp($::operatingsystemmajrelease, '7') >= 0) { $iscsi_helper = 'lioadm' diff --git a/manila/spec/acceptance/basic_manila_spec.rb b/manila/spec/acceptance/basic_manila_spec.rb index 519496b87..7fb6b3fde 100644 --- a/manila/spec/acceptance/basic_manila_spec.rb +++ b/manila/spec/acceptance/basic_manila_spec.rb @@ -29,7 +29,7 @@ # Manila resources class { '::manila': - sql_connection => 'mysql://manila:a_big_secret@127.0.0.1/manila?charset=utf8', + sql_connection => 'mysql+pymysql://manila:a_big_secret@127.0.0.1/manila?charset=utf8', rabbit_userid => 'manila', rabbit_password => 'an_even_bigger_secret', rabbit_host => '127.0.0.1', diff --git a/manila/spec/classes/manila_db_spec.rb b/manila/spec/classes/manila_db_spec.rb index 7f75fac1c..39c3dabc0 100644 --- a/manila/spec/classes/manila_db_spec.rb +++ b/manila/spec/classes/manila_db_spec.rb @@ -18,7 +18,7 @@ context 'with specific parameters' do let :params do - { :database_connection => 'mysql://manila:manila@localhost/manila', + { :database_connection => 'mysql+pymysql://manila:manila@localhost/manila', :database_idle_timeout => '3601', :database_min_pool_size => '2', :database_max_pool_size => '21', @@ -27,7 +27,7 @@ :database_retry_interval => '11', } end - it { is_expected.to contain_manila_config('database/connection').with_value('mysql://manila:manila@localhost/manila').with_secret(true) } + it { is_expected.to contain_manila_config('database/connection').with_value('mysql+pymysql://manila:manila@localhost/manila').with_secret(true) } it { is_expected.to contain_manila_config('database/idle_timeout').with_value('3601') } it { is_expected.to contain_manila_config('database/min_pool_size').with_value('2') } it { is_expected.to contain_manila_config('database/max_retries').with_value('11') } @@ -37,6 +37,14 @@ end + context 'with MySQL-python library as backend package' do + let :params do + { :database_connection => 'mysql://manila:manila@localhost/manila' } + end + + it { is_expected.to contain_manila_config('database/connection').with_value('mysql://manila:manila@localhost/manila').with_secret(true) } + end + context 'with postgresql backend' do let :params do { :database_connection => 'postgresql://manila:manila@localhost/manila', } @@ -56,6 +64,14 @@ it_raises 'a Puppet::Error', /validate_re/ end + context 'with incorrect database_connection string' do + let :params do + { :database_connection => 'foo+pymysql://manila:manila@localhost/manila', } + end + + it_raises 'a Puppet::Error', /validate_re/ + end + end context 'on Debian platforms' do @@ -67,6 +83,14 @@ end it_configures 'manila::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://manila:manila@localhost/manila' } + end + + it { is_expected.to contain_package('manila-backend-package').with({ :ensure => 'present', :name => 'python-pymysql' }) } + end end context 'on Redhat platforms' do @@ -77,6 +101,14 @@ end it_configures 'manila::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://manila:manila@localhost/manila' } + end + + it { is_expected.not_to contain_package('manila-backend-package') } + end end end diff --git a/manila/spec/classes/manila_keystone_auth_spec.rb b/manila/spec/classes/manila_keystone_auth_spec.rb index 27acfb6db..2027e54c3 100644 --- a/manila/spec/classes/manila_keystone_auth_spec.rb +++ b/manila/spec/classes/manila_keystone_auth_spec.rb @@ -20,9 +20,8 @@ :ensure => 'present', :roles => ['admin'] ) - is_expected.to contain_keystone_service('manila').with( + is_expected.to contain_keystone_service('manila::share').with( :ensure => 'present', - :type => 'share', :description => 'Manila Service' ) @@ -35,20 +34,19 @@ :ensure => 'present', :roles => ['admin'] ) - is_expected.to contain_keystone_service('manilav2').with( + is_expected.to contain_keystone_service('manilav2::sharev2').with( :ensure => 'present', - :type => 'sharev2', :description => 'Manila Service v2' ) end - it { is_expected.to contain_keystone_endpoint('RegionOne/manila').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/manila::share').with( :ensure => 'present', :public_url => 'http://127.0.0.1:8786/v1/%(tenant_id)s', :admin_url => 'http://127.0.0.1:8786/v1/%(tenant_id)s', :internal_url => 'http://127.0.0.1:8786/v1/%(tenant_id)s' ) } - it { is_expected.to contain_keystone_endpoint('RegionOne/manilav2').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/manilav2::sharev2').with( :ensure => 'present', :public_url => 'http://127.0.0.1:8786/v2/%(tenant_id)s', :admin_url => 'http://127.0.0.1:8786/v2/%(tenant_id)s', @@ -70,13 +68,13 @@ ) end - it { is_expected.to contain_keystone_endpoint('RegionThree/manila').with( + it { is_expected.to contain_keystone_endpoint('RegionThree/manila::share').with( :ensure => 'present', :public_url => 'https://10.0.42.1:4242/v42/%(tenant_id)s', :admin_url => 'https://10.0.42.2:4242/v42/%(tenant_id)s', :internal_url => 'https://10.0.42.3:4242/v42/%(tenant_id)s' )} - it { is_expected.to contain_keystone_endpoint('RegionThree/manilav2').with( + it { is_expected.to contain_keystone_endpoint('RegionThree/manilav2::sharev2').with( :ensure => 'present', :public_url => 'https://10.0.42.1:4242/v43/%(tenant_id)s', :admin_url => 'https://10.0.42.2:4242/v43/%(tenant_id)s', @@ -99,7 +97,7 @@ ) end - it { is_expected.to contain_keystone_endpoint('RegionThree/manila').with( + it { is_expected.to contain_keystone_endpoint('RegionThree/manila::share').with( :ensure => 'present', :public_url => 'https://10.0.42.1:4242/v42/%(tenant_id)s', :admin_url => 'https://10.0.42.2:4242/v42/%(tenant_id)s', @@ -114,7 +112,7 @@ :configure_endpoint_v2 => false ) end - it { is_expected.to_not contain_keystone_endpoint('RegionOne/manila') } - it { is_expected.to_not contain_keystone_endpoint('RegionOne/manilav2') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/manila::share') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/manilav2::sharev2') } end end diff --git a/manila/spec/classes/manila_spec.rb b/manila/spec/classes/manila_spec.rb index 19e6b9c76..1973e1fbb 100644 --- a/manila/spec/classes/manila_spec.rb +++ b/manila/spec/classes/manila_spec.rb @@ -1,7 +1,7 @@ require 'spec_helper' describe 'manila' do let :req_params do - {:rabbit_password => 'guest', :sql_connection => 'mysql://user:password@host/database'} + {:rabbit_password => 'guest', :sql_connection => 'mysql+pymysql://user:password@host/database'} end let :facts do @@ -317,7 +317,7 @@ let :params do { - :sql_connection => 'mysql://user:password@host/database', + :sql_connection => 'mysql+pymysql://user:password@host/database', :rpc_backend => 'zmq', } end diff --git a/neutron/manifests/db.pp b/neutron/manifests/db.pp index 1b013e88b..c5b1ad9d8 100644 --- a/neutron/manifests/db.pp +++ b/neutron/manifests/db.pp @@ -43,6 +43,8 @@ $database_max_overflow = 20, ) { + include ::neutron::params + # NOTE(spredzy): In order to keep backward compatibility we rely on the pick function # to use neutron:: if neutron::db:: isn't specified. $database_connection_real = pick($::neutron::server::database_connection, $database_connection) @@ -54,14 +56,18 @@ $database_max_overflow_real = pick($::neutron::server::database_max_overflow, $database_max_overflow) validate_re($database_connection_real, - '(sqlite|mysql|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') + '^(sqlite|mysql(\+pymysql)?|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') if $database_connection_real { case $database_connection_real { - /^mysql:\/\//: { - $backend_package = false + /^mysql(\+pymysql)?:\/\//: { require 'mysql::bindings' require 'mysql::bindings::python' + if $database_connection_real =~ /^mysql\+pymysql/ { + $backend_package = $::neutron::params::pymysql_package_name + } else { + $backend_package = false + } } /^postgresql:\/\//: { $backend_package = false diff --git a/neutron/manifests/keystone/auth.pp b/neutron/manifests/keystone/auth.pp index d7cd7bc5b..abf7655ff 100644 --- a/neutron/manifests/keystone/auth.pp +++ b/neutron/manifests/keystone/auth.pp @@ -189,7 +189,7 @@ $real_service_name = pick($service_name, $auth_name) if $configure_endpoint { - Keystone_endpoint["${region}/${real_service_name}"] ~> Service <| title == 'neutron-server' |> + Keystone_endpoint["${region}/${real_service_name}::${service_type}"] ~> Service <| title == 'neutron-server' |> } if $configure_user_role { diff --git a/neutron/manifests/params.pp b/neutron/manifests/params.pp index 47266c216..7f7f9a146 100644 --- a/neutron/manifests/params.pp +++ b/neutron/manifests/params.pp @@ -78,6 +78,7 @@ $kernel_headers = "linux-headers-${::kernelrelease}" $sqlite_package_name = undef + $pymysql_package_name = undef } elsif($::osfamily == 'Debian') { @@ -159,6 +160,7 @@ $kernel_headers = "linux-headers-${::kernelrelease}" $sqlite_package_name = 'python-pysqlite2' + $pymysql_package_name = 'python-pymysql' } else { fail("Unsupported osfamily ${::osfamily}") diff --git a/neutron/manifests/plugins/ml2/opendaylight.pp b/neutron/manifests/plugins/ml2/opendaylight.pp new file mode 100644 index 000000000..a8945fd8c --- /dev/null +++ b/neutron/manifests/plugins/ml2/opendaylight.pp @@ -0,0 +1,60 @@ +# +# Install the OpenDaylight and generate config file +# from parameters in the other classes. +# +# === Parameters +# +# [*package_ensure*] +# (optional) The intended state of the python-networking-odl +# package, i.e. any of the possible values of the 'ensure' +# property for a package resource type. +# Defaults to 'present' +# +# [*odl_username*] +# (optional) The opendaylight controller username +# Defaults to undef +# Example: 'admin' +# +# [*odl_password*] +# (optional) The opendaylight controller password +# Defaults to undef +# Example: 'admin' +# +# [*odl_url*] +# (optional) The opendaylight controller neutron URL +# Defaults to undef +# Example: 'http://127.0.0.1:8080/controller/nb/v2/neutron' +# +class neutron::plugins::ml2::opendaylight ( + $package_ensure = 'present', + $odl_username = undef, + $odl_password = undef, + $odl_url = undef +) { + require ::neutron::plugins::ml2 + + ensure_resource('package', 'python-networking-odl', + { + ensure => $package_ensure, + tag => 'openstack', + } + ) + + if ($odl_username) { + neutron_plugin_ml2 { 'ml2_odl/username': value => $odl_username; } + } else { + neutron_plugin_ml2 { 'ml2_odl/username': ensure => absent; } + } + + if ($odl_password) { + neutron_plugin_ml2 { 'ml2_odl/password': value => $odl_password; } + } else { + neutron_plugin_ml2 { 'ml2_odl/password': ensure => absent; } + } + + if ($odl_url) { + neutron_plugin_ml2 { 'ml2_odl/url': value => $odl_url; } + } else { + neutron_plugin_ml2 { 'ml2_odl/url': ensure => absent; } + } +} diff --git a/neutron/manifests/plugins/ovs/opendaylight.pp b/neutron/manifests/plugins/ovs/opendaylight.pp new file mode 100644 index 000000000..8a961c082 --- /dev/null +++ b/neutron/manifests/plugins/ovs/opendaylight.pp @@ -0,0 +1,79 @@ +# +# Configure OVS to use OpenDaylight +# +# === Parameters +# +# [*tunnel_ip*] +# (required) The IP of the host to use for tunneling +# tenant VXLAN/GRE over +# +# [*odl_username*] +# (optional) The opendaylight controller username +# Defaults to 'admin' +# +# [*odl_password*] +# (optional) The opendaylight controller password +# Defaults to 'admin' +# +# [*odl_check_url*] +# (optional) The URL used to check ODL is available and ready +# Defaults to 'http://127.0.0.1:8080/restconf/operational/network-topology:network-topology/topology/netvirt:1' +# +# [*odl_ovsdb_iface*] +# (optional) The ODL southbound interface for OVSDB +# Defaults to 'tcp:127.0.0.1:6640' +# +# [*provider_mappings*] +# (optional) bridge mappings required if using VLAN +# tenant type. Example: provider_mappings=br-ex:eth0 +# Defaults to false +# +# [*retry_interval*] +# (optional) The time (in seconds) to wait between ODL availability checks +# Defaults to 60 +# +# [*retry_count*] +# (optional) The number of ODL availability checks to run before failing +# Defaults to 20 +# +class neutron::plugins::ovs::opendaylight ( + $tunnel_ip, + $odl_username = 'admin', + $odl_password = 'admin', + $odl_check_url = 'http://127.0.0.1:8080/restconf/operational/network-topology:network-topology/topology/netvirt:1', + $odl_ovsdb_iface = 'tcp:127.0.0.1:6640', + $provider_mappings = false, + $retry_interval = 60, + $retry_count = 20, +) { + # Handle the case where ODL controller is also on this host + Service<| title == 'opendaylight' |> -> Exec <| title == 'Wait for NetVirt OVSDB to come up' |> + + exec { 'Wait for NetVirt OVSDB to come up': + command => "curl -o /dev/null --fail --silent --head -u ${odl_username}:${odl_password} ${odl_check_url}", + tries => $retry_count, + try_sleep => $retry_interval, + path => '/usr/sbin:/usr/bin:/sbin:/bin', + } -> + # OVS manager + exec { 'Set OVS Manager to OpenDaylight': + command => "ovs-vsctl set-manager ${odl_ovsdb_iface}", + unless => "ovs-vsctl show | grep 'Manager \"${odl_ovsdb_iface}\"'", + path => '/usr/sbin:/usr/bin:/sbin:/bin', + } -> + # local ip + exec { 'Set local_ip Other Option': + command => "ovs-vsctl set Open_vSwitch $(ovs-vsctl get Open_vSwitch . _uuid) other_config:local_ip=${tunnel_ip}", + unless => "ovs-vsctl list Open_vSwitch | grep 'local_ip=\"${tunnel_ip}\"'", + path => '/usr/sbin:/usr/bin:/sbin:/bin', + } + + # set mappings for VLAN + if $provider_mappings { + exec { 'Set provider_mappings Other Option': + command => "ovs-vsctl set Open_vSwitch $(ovs-vsctl get Open_vSwitch . _uuid) other_config:provider_mappings=${provider_mappings}", + unless => "ovs-vsctl list Open_vSwitch | grep 'provider_mappings' | grep ${provider_mappings}", + path => '/usr/sbin:/usr/bin:/sbin:/bin', + } + } +} diff --git a/neutron/manifests/services/fwaas.pp b/neutron/manifests/services/fwaas.pp index acd933b8c..f944a47bd 100644 --- a/neutron/manifests/services/fwaas.pp +++ b/neutron/manifests/services/fwaas.pp @@ -45,6 +45,9 @@ include ::neutron::params + # FWaaS needs to be enabled before starting Neutron L3 agent + Neutron_fwaas_service_config<||> ~> Service['neutron-l3'] + if ($::osfamily == 'Debian') { # Debian platforms if $vpnaas_agent_package { diff --git a/neutron/spec/acceptance/basic_neutron_spec.rb b/neutron/spec/acceptance/basic_neutron_spec.rb index cdec97e72..2b17e4cd5 100644 --- a/neutron/spec/acceptance/basic_neutron_spec.rb +++ b/neutron/spec/acceptance/basic_neutron_spec.rb @@ -49,7 +49,7 @@ class { '::neutron::keystone::auth': password => 'a_big_secret', } class { '::neutron::server': - database_connection => 'mysql://neutron:a_big_secret@127.0.0.1/neutron?charset=utf8', + database_connection => 'mysql+pymysql://neutron:a_big_secret@127.0.0.1/neutron?charset=utf8', auth_password => 'a_big_secret', identity_uri => 'http://127.0.0.1:35357/', sync_db => true, diff --git a/neutron/spec/classes/neutron_db_spec.rb b/neutron/spec/classes/neutron_db_spec.rb index 6c1b1a02f..806bf95a0 100644 --- a/neutron/spec/classes/neutron_db_spec.rb +++ b/neutron/spec/classes/neutron_db_spec.rb @@ -16,14 +16,14 @@ context 'with specific parameters' do let :params do - { :database_connection => 'mysql://neutron:neutron@localhost/neutron', + { :database_connection => 'mysql+pymysql://neutron:neutron@localhost/neutron', :database_idle_timeout => '3601', :database_min_pool_size => '2', :database_max_retries => '11', :database_retry_interval => '11', } end - it { is_expected.to contain_neutron_config('database/connection').with_value('mysql://neutron:neutron@localhost/neutron').with_secret(true) } + it { is_expected.to contain_neutron_config('database/connection').with_value('mysql+pymysql://neutron:neutron@localhost/neutron').with_secret(true) } it { is_expected.to contain_neutron_config('database/idle_timeout').with_value('3601') } it { is_expected.to contain_neutron_config('database/min_pool_size').with_value('2') } it { is_expected.to contain_neutron_config('database/max_retries').with_value('11') } @@ -31,6 +31,14 @@ end + context 'with MySQL-python library as backend package' do + let :params do + { :database_connection => 'mysql://neutron:neutron@localhost/neutron' } + end + + it { is_expected.to contain_neutron_config('database/connection').with_value('mysql://neutron:neutron@localhost/neutron').with_secret(true) } + end + context 'with postgresql backend' do let :params do { :database_connection => 'postgresql://neutron:neutron@localhost/neutron', } @@ -50,6 +58,14 @@ it_raises 'a Puppet::Error', /validate_re/ end + context 'with incorrect database_connection string' do + let :params do + { :database_connection => 'foo+pymysql://neutron:neutron@localhost/neutron', } + end + + it_raises 'a Puppet::Error', /validate_re/ + end + end context 'on Debian platforms' do @@ -61,6 +77,15 @@ end it_configures 'neutron::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://neutron:neutron@localhost/neutron' } + end + + it { is_expected.to contain_package('neutron-backend-package').with({ :ensure => 'present', :name => 'python-pymysql' }) } + end + end context 'on Redhat platforms' do @@ -71,6 +96,15 @@ end it_configures 'neutron::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://neutron:neutron@localhost/neutron' } + end + + it { is_expected.not_to contain_package('neutron-backend-package') } + end + end end diff --git a/neutron/spec/classes/neutron_keystone_auth_spec.rb b/neutron/spec/classes/neutron_keystone_auth_spec.rb index 8da3fcbb6..fb90b9eeb 100644 --- a/neutron/spec/classes/neutron_keystone_auth_spec.rb +++ b/neutron/spec/classes/neutron_keystone_auth_spec.rb @@ -26,13 +26,12 @@ :roles => ['admin'] )} - it { is_expected.to contain_keystone_service('neutron').with( + it { is_expected.to contain_keystone_service('neutron::network').with( :ensure => 'present', - :type => 'network', :description => 'Neutron Networking Service' ) } - it { is_expected.to contain_keystone_endpoint('RegionOne/neutron').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/neutron::network').with( :ensure => 'present', :public_url => "http://127.0.0.1:9696", :admin_url => "http://127.0.0.1:9696", @@ -57,7 +56,7 @@ } end - it { is_expected.to contain_keystone_endpoint('RegionOne/neutron').with_notify(['Service[neutron-server]']) } + it { is_expected.to contain_keystone_endpoint('RegionOne/neutron::network').with_notify(['Service[neutron-server]']) } end describe 'with endpoint URL parameters' do @@ -70,7 +69,7 @@ } end - it { is_expected.to contain_keystone_endpoint('RegionOne/neutron').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/neutron::network').with( :ensure => 'present', :public_url => 'https://10.10.10.10:80', :internal_url => 'https://10.10.10.11:81', @@ -93,7 +92,7 @@ } end - it { is_expected.to contain_keystone_endpoint('RegionOne/neutron').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/neutron::network').with( :ensure => 'present', :public_url => "https://10.10.10.10:80", :internal_url => "https://10.10.10.11:81", @@ -114,9 +113,9 @@ it { is_expected.to contain_keystone_user_role('neutrony@services') } - it { is_expected.to contain_keystone_service('neutrony') } + it { is_expected.to contain_keystone_service('neutrony::network') } - it { is_expected.to contain_keystone_endpoint('RegionOne/neutrony') } + it { is_expected.to contain_keystone_endpoint('RegionOne/neutrony::network') } end @@ -131,8 +130,8 @@ it { is_expected.to contain_keystone_user('neutron') } it { is_expected.to contain_keystone_user_role('neutron@services') } - it { is_expected.to contain_keystone_service('neutron_service') } - it { is_expected.to contain_keystone_endpoint('RegionOne/neutron_service') } + it { is_expected.to contain_keystone_service('neutron_service::network') } + it { is_expected.to contain_keystone_endpoint('RegionOne/neutron_service::network') } end @@ -149,9 +148,8 @@ it { is_expected.to contain_keystone_user_role('neutron@services') } - it { is_expected.to contain_keystone_service('neutron').with( + it { is_expected.to contain_keystone_service('neutron::network').with( :ensure => 'present', - :type => 'network', :description => 'Neutron Networking Service' ) } @@ -171,9 +169,8 @@ it { is_expected.not_to contain_keystone_user_role('neutron@services') } - it { is_expected.to contain_keystone_service('neutron').with( + it { is_expected.to contain_keystone_service('neutron::network').with( :ensure => 'present', - :type => 'network', :description => 'Neutron Networking Service' ) } @@ -188,7 +185,7 @@ } end - it { is_expected.to_not contain_keystone_endpoint('RegionOne/neutron') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/neutron::network') } end diff --git a/neutron/spec/classes/neutron_plugins_ml2_opendaylight_spec.rb b/neutron/spec/classes/neutron_plugins_ml2_opendaylight_spec.rb new file mode 100644 index 000000000..6eefa6d17 --- /dev/null +++ b/neutron/spec/classes/neutron_plugins_ml2_opendaylight_spec.rb @@ -0,0 +1,73 @@ +require 'spec_helper' + +describe 'neutron::plugins::ml2::opendaylight' do + + let :pre_condition do + "class { 'neutron::server': auth_password => 'password'} + class { 'neutron': + rabbit_password => 'passw0rd', + core_plugin => 'neutron.plugins.ml2.plugin.Ml2Plugin' }" + end + + let :default_params do + { + :package_ensure => 'present', + #:odl_username => '', + #:odl_password => '', + #:odl_url => '', + } + end + + let :params do + { + } + end + + let :test_facts do + { + :operatingsystem => 'default', + :operatingsystemrelease => 'default', + } + end + + + shared_examples_for 'neutron plugin opendaylight ml2' do + before do + params.merge!(default_params) + end + + it 'should have' do + is_expected.to contain_package('python-networking-odl').with( + :ensure => params[:package_ensure], + :tag => 'openstack' + ) + end + + it 'configures ml2_odl settings' do + is_expected.to contain_neutron_plugin_ml2('ml2_odl/password').with_ensure('absent') + is_expected.to contain_neutron_plugin_ml2('ml2_odl/username').with_ensure('absent') + is_expected.to contain_neutron_plugin_ml2('ml2_odl/url').with_ensure('absent') + end + end + + context 'on RedHat platforms' do + let :facts do + test_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) + end + + it_configures 'neutron plugin opendaylight ml2' + end + + context 'on Debian platforms' do + let :facts do + test_facts.merge({ + :osfamily => 'Debian', + }) + end + + it_configures 'neutron plugin opendaylight ml2' + end +end diff --git a/neutron/spec/classes/neutron_plugins_ovs_opendaylight_spec.rb b/neutron/spec/classes/neutron_plugins_ovs_opendaylight_spec.rb new file mode 100644 index 000000000..10dd7082f --- /dev/null +++ b/neutron/spec/classes/neutron_plugins_ovs_opendaylight_spec.rb @@ -0,0 +1,88 @@ +describe 'neutron::plugins::ovs::opendaylight' do + + let :pre_condition do + "class { 'neutron::server': auth_password => 'password'} + class { 'neutron': + rabbit_password => 'passw0rd', + core_plugin => 'neutron.plugins.ml2.plugin.Ml2Plugin' }" + end + + let :default_params do + { + :odl_username => 'admin', + :odl_password => 'admin', + :odl_check_url => 'http://127.0.0.1:8080/restconf/operational/network-topology:network-topology/topology/netvirt:1', + :odl_ovsdb_iface => 'tcp:127.0.0.1:6640', + :provider_mappings => false, + :retry_interval => 60, + :retry_count => 20, + } + end + + let :params do + { + :tunnel_ip => '127.0.0.1', + } + end + + let :test_facts do + { + :operatingsystem => 'default', + :operatingsystemrelease => 'default', + } + end + + + shared_examples_for 'neutron plugin opendaylight ovs' do + before do + params.merge!(default_params) + end + + context 'with provider mappings' do + before do + params.merge!({ :provider_mappings => true }) + end + it_configures 'with provider mappings' + end + it_configures 'with default parameters' + end + + shared_examples_for 'with default parameters' do + it 'configures OVS for ODL' do + is_expected.to contain_exec('Wait for NetVirt OVSDB to come up') + is_expected.to contain_exec('Set OVS Manager to OpenDaylight') + is_expected.to contain_exec('Set local_ip Other Option') + is_expected.not_to contain_exec('Set provider_mappings Other Option') + end + end + + shared_examples_for 'with provider mappings' do + it 'configures OVS for ODL' do + is_expected.to contain_exec('Wait for NetVirt OVSDB to come up') + is_expected.to contain_exec('Set OVS Manager to OpenDaylight') + is_expected.to contain_exec('Set local_ip Other Option') + is_expected.to contain_exec('Set provider_mappings Other Option') + end + end + + context 'on RedHat platforms' do + let :facts do + test_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + }) + end + + it_configures 'neutron plugin opendaylight ovs' + end + + context 'on Debian platforms' do + let :facts do + test_facts.merge({ + :osfamily => 'Debian' + }) + end + + it_configures 'neutron plugin opendaylight ovs' + end +end diff --git a/neutron/spec/classes/neutron_services_fwaas_spec.rb b/neutron/spec/classes/neutron_services_fwaas_spec.rb index ae13d5330..f223b7d34 100644 --- a/neutron/spec/classes/neutron_services_fwaas_spec.rb +++ b/neutron/spec/classes/neutron_services_fwaas_spec.rb @@ -22,7 +22,8 @@ describe 'neutron::services::fwaas' do let :pre_condition do - "class { 'neutron': rabbit_password => 'passw0rd' }" + "class { 'neutron': rabbit_password => 'passw0rd' } + include ::neutron::agents::l3" end let :default_facts do diff --git a/nova/manifests/db.pp b/nova/manifests/db.pp index 5fb320137..65144ef0e 100644 --- a/nova/manifests/db.pp +++ b/nova/manifests/db.pp @@ -81,13 +81,17 @@ if $database_connection_real { validate_re($database_connection_real, - '(sqlite|mysql|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') + '^(sqlite|mysql(\+pymysql)?|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') case $database_connection_real { - /^mysql:\/\//: { - $backend_package = false + /^mysql(\+pymysql)?:\/\//: { require 'mysql::bindings' require 'mysql::bindings::python' + if $database_connection_real =~ /^mysql\+pymysql/ { + $backend_package = $::nova::params::pymysql_package_name + } else { + $backend_package = false + } } /^postgresql:\/\//: { $backend_package = false diff --git a/nova/manifests/keystone/auth.pp b/nova/manifests/keystone/auth.pp index 25f7d2f11..893e1bb9e 100644 --- a/nova/manifests/keystone/auth.pp +++ b/nova/manifests/keystone/auth.pp @@ -317,7 +317,7 @@ } if $configure_endpoint { - Keystone_endpoint["${region}/${real_service_name}"] ~> Service <| name == 'nova-api' |> + Keystone_endpoint["${region}/${real_service_name}::compute"] ~> Service <| name == 'nova-api' |> } keystone::resource::service_identity { "nova service, user ${auth_name}": diff --git a/nova/manifests/params.pp b/nova/manifests/params.pp index 6c9628cd0..29e9b0f95 100644 --- a/nova/manifests/params.pp +++ b/nova/manifests/params.pp @@ -27,6 +27,7 @@ $serialproxy_package_name = 'openstack-nova-serialproxy' $spicehtml5proxy_package_name = 'openstack-nova-console' $sqlite_package_name = undef + $pymysql_package_name = 'python2-PyMySQL' # service names $api_service_name = 'openstack-nova-api' $cells_service_name = 'openstack-nova-cells' @@ -82,6 +83,7 @@ $tgt_package_name = 'tgt' $serialproxy_package_name = 'nova-serialproxy' $sqlite_package_name = 'python-pysqlite2' + $pymysql_package_name = 'python-pymysql' # service names $api_service_name = 'nova-api' $cells_service_name = 'nova-cells' diff --git a/nova/spec/acceptance/basic_nova_spec.rb b/nova/spec/acceptance/basic_nova_spec.rb index 0465bdfa9..644889852 100644 --- a/nova/spec/acceptance/basic_nova_spec.rb +++ b/nova/spec/acceptance/basic_nova_spec.rb @@ -29,7 +29,7 @@ # Nova resources class { '::nova': - database_connection => 'mysql://nova:a_big_secret@127.0.0.1/nova?charset=utf8', + database_connection => 'mysql+pymysql://nova:a_big_secret@127.0.0.1/nova?charset=utf8', rabbit_userid => 'nova', rabbit_password => 'an_even_bigger_secret', image_service => 'nova.image.glance.GlanceImageService', diff --git a/nova/spec/classes/nova_db_spec.rb b/nova/spec/classes/nova_db_spec.rb index 9c841cb58..2a76aa0c5 100644 --- a/nova/spec/classes/nova_db_spec.rb +++ b/nova/spec/classes/nova_db_spec.rb @@ -20,17 +20,18 @@ context 'with overriden parameters' do before :each do params.merge!( - :database_connection => 'mysql://user:pass@db/db', - :slave_connection => 'mysql://user:pass@slave/db', + :database_connection => 'mysql+pymysql://user:pass@db/db', + :slave_connection => 'mysql+pymysql://user:pass@slave/db', ) end - it { is_expected.to contain_nova_config('database/connection').with_value('mysql://user:pass@db/db').with_secret(true) } - it { is_expected.to contain_nova_config('database/slave_connection').with_value('mysql://user:pass@slave/db').with_secret(true) } + it { is_expected.to contain_nova_config('database/connection').with_value('mysql+pymysql://user:pass@db/db').with_secret(true) } + it { is_expected.to contain_nova_config('database/slave_connection').with_value('mysql+pymysql://user:pass@slave/db').with_secret(true) } it { is_expected.to contain_nova_config('database/idle_timeout').with_value('3600') } it { is_expected.to contain_nova_config('database/min_pool_size').with_value('1') } it { is_expected.to contain_nova_config('database/max_retries').with_value('10') } it { is_expected.to contain_nova_config('database/retry_interval').with_value('10') } + it { is_expected.to contain_package('nova-backend-package').with({ :ensure => 'present', :name => platform_params[:pymysql_package_name] }) } end @@ -45,6 +46,14 @@ end + context 'with MySQL-python library as backend package' do + let :params do + { :database_connection => 'mysql://user:pass@db/db', } + end + + it { is_expected.to contain_package('python-mysqldb').with(:ensure => 'present') } + end + context 'with incorrect database_connection string' do let :params do { :database_connection => 'redis://nova:nova@localhost/nova', } @@ -53,6 +62,13 @@ it_raises 'a Puppet::Error', /validate_re/ end + context 'with incorrect pymysql database_connection string' do + let :params do + { :database_connection => 'foo+pymysql://user:pass@db/db', } + end + + it_raises 'a Puppet::Error', /validate_re/ + end end context 'on Debian platforms' do @@ -63,6 +79,10 @@ } end + let :platform_params do + { :pymysql_package_name => 'python-pymysql' } + end + it_configures 'nova::db' context 'with sqlite backend' do @@ -88,6 +108,10 @@ } end + let :platform_params do + { :pymysql_package_name => 'python2-PyMySQL' } + end + it_configures 'nova::db' end diff --git a/nova/spec/classes/nova_keystone_auth_spec.rb b/nova/spec/classes/nova_keystone_auth_spec.rb index 5da3568e5..b4e9176ef 100644 --- a/nova/spec/classes/nova_keystone_auth_spec.rb +++ b/nova/spec/classes/nova_keystone_auth_spec.rb @@ -35,39 +35,36 @@ :roles => ['admin'] )} - it { is_expected.to contain_keystone_service('nova').with( - :ensure => 'present', - :type => 'compute', + it { is_expected.to contain_keystone_service('nova::compute').with( + :ensure => 'present', :description => 'Openstack Compute Service' )} - it { is_expected.to contain_keystone_service('novav3').with( - :ensure => 'present', - :type => 'computev3', + it { is_expected.to contain_keystone_service('novav3::computev3').with( + :ensure => 'present', :description => 'Openstack Compute Service v3' )} - it { is_expected.to contain_keystone_service('nova_ec2').with( - :ensure => 'present', - :type => 'ec2', + it { is_expected.to contain_keystone_service('nova_ec2::ec2').with( + :ensure => 'present', :description => 'EC2 Service' )} - it { is_expected.to contain_keystone_endpoint('RegionOne/nova').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/nova::compute').with( :ensure => 'present', :public_url => 'http://127.0.0.1:8774/v2/%(tenant_id)s', :admin_url => 'http://127.0.0.1:8774/v2/%(tenant_id)s', :internal_url => 'http://127.0.0.1:8774/v2/%(tenant_id)s' )} - it { is_expected.to contain_keystone_endpoint('RegionOne/novav3').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/novav3::computev3').with( :ensure => 'present', :public_url => 'http://127.0.0.1:8774/v3', :admin_url => 'http://127.0.0.1:8774/v3', :internal_url => 'http://127.0.0.1:8774/v3' )} - it { is_expected.to contain_keystone_endpoint('RegionOne/nova_ec2').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/nova_ec2::ec2').with( :ensure => 'present', :public_url => 'http://127.0.0.1:8773/services/Cloud', :admin_url => 'http://127.0.0.1:8773/services/Admin', @@ -91,15 +88,13 @@ :roles => ['admin'] )} - it { is_expected.to contain_keystone_service('foo').with( + it { is_expected.to contain_keystone_service('foo::compute').with( :ensure => 'present', - :type => 'compute', :description => 'Openstack Compute Service' )} - it { is_expected.to contain_keystone_service('foo_ec2').with( + it { is_expected.to contain_keystone_service('foo_ec2::ec2').with( :ensure => 'present', - :type => 'ec2', :description => 'EC2 Service' )} @@ -117,8 +112,8 @@ it { is_expected.to contain_keystone_user('thesame').with(:ensure => 'present') } it { is_expected.to contain_keystone_user_role('thesame@services').with(:ensure => 'present') } - it { is_expected.to contain_keystone_service('nova').with(:ensure => 'present') } - it { is_expected.to contain_keystone_service('novav3').with(:ensure => 'present') } + it { is_expected.to contain_keystone_service('nova::compute').with(:ensure => 'present') } + it { is_expected.to contain_keystone_service('novav3::computev3').with(:ensure => 'present') } end @@ -145,7 +140,7 @@ end it do - expect { is_expected.to contain_keystone_service('nova') }.to raise_error(Puppet::Error, /service_name and service_name_v3 must be different/) + expect { is_expected.to contain_keystone_service('nova::compute') }.to raise_error(Puppet::Error, /service_name and service_name_v3 must be different/) end end @@ -166,21 +161,21 @@ ) end - it { is_expected.to contain_keystone_endpoint('RegionTwo/nova').with( + it { is_expected.to contain_keystone_endpoint('RegionTwo/nova::compute').with( :ensure => 'present', :public_url => params[:public_url], :internal_url => params[:internal_url], :admin_url => params[:admin_url] )} - it { is_expected.to contain_keystone_endpoint('RegionTwo/novav3').with( + it { is_expected.to contain_keystone_endpoint('RegionTwo/novav3::computev3').with( :ensure => 'present', :public_url => params[:public_url_v3], :internal_url => params[:internal_url_v3], :admin_url => params[:admin_url_v3] )} - it { is_expected.to contain_keystone_endpoint('RegionTwo/nova_ec2').with( + it { is_expected.to contain_keystone_endpoint('RegionTwo/nova_ec2::ec2').with( :ensure => 'present', :public_url => params[:ec2_public_url], :internal_url => params[:ec2_internal_url], @@ -204,14 +199,14 @@ ) end - it { is_expected.to contain_keystone_endpoint('RegionTwo/nova').with( + it { is_expected.to contain_keystone_endpoint('RegionTwo/nova::compute').with( :ensure => 'present', :public_url => 'https://10.0.0.1:9774/v2.2/%(tenant_id)s', :admin_url => 'https://10.0.0.2:9774/v2.2/%(tenant_id)s', :internal_url => 'https://10.0.0.3:9774/v2.2/%(tenant_id)s' )} - it { is_expected.to contain_keystone_endpoint('RegionTwo/nova_ec2').with( + it { is_expected.to contain_keystone_endpoint('RegionTwo/nova_ec2::ec2').with( :ensure => 'present', :public_url => 'https://10.0.0.1:9773/services/Cloud', :admin_url => 'https://10.0.0.2:9773/services/Admin', @@ -224,7 +219,7 @@ params.merge!( :configure_endpoint => false ) end - it { is_expected.to_not contain_keystone_endpoint('RegionOne/nova') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/nova::compute') } end describe 'when disabling EC2 endpoint' do @@ -232,8 +227,8 @@ params.merge!( :configure_ec2_endpoint => false ) end - it { is_expected.to_not contain_keystone_service('nova_ec2') } - it { is_expected.to_not contain_keystone_endpoint('RegionOne/nova_ec2') } + it { is_expected.to_not contain_keystone_service('nova_ec2::ec2') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/nova_ec2::ec2') } end describe 'when disabling user configuration' do @@ -243,9 +238,8 @@ it { is_expected.to_not contain_keystone_user('nova') } it { is_expected.to contain_keystone_user_role('nova@services') } - it { is_expected.to contain_keystone_service('nova').with( - :ensure => 'present', - :type => 'compute', + it { is_expected.to contain_keystone_service('nova::compute').with( + :ensure => 'present', :description => 'Openstack Compute Service' )} end @@ -261,9 +255,8 @@ it { is_expected.to_not contain_keystone_user('nova') } it { is_expected.to_not contain_keystone_user_role('nova@services') } - it { is_expected.to contain_keystone_service('nova').with( - :ensure => 'present', - :type => 'compute', + it { is_expected.to contain_keystone_service('nova::compute').with( + :ensure => 'present', :description => 'Openstack Compute Service' )} end @@ -284,7 +277,7 @@ } end - it { is_expected.to contain_keystone_endpoint('RegionOne/nova').with_notify(['Service[nova-api]']) } + it { is_expected.to contain_keystone_endpoint('RegionOne/nova::compute').with_notify(['Service[nova-api]']) } end describe 'when overriding service names' do @@ -299,12 +292,12 @@ it { is_expected.to contain_keystone_user('nova') } it { is_expected.to contain_keystone_user_role('nova@services') } - it { is_expected.to contain_keystone_service('nova_service') } - it { is_expected.to contain_keystone_service('nova_service_v3') } - it { is_expected.to contain_keystone_service('nova_service_ec2') } - it { is_expected.to contain_keystone_endpoint('RegionOne/nova_service') } - it { is_expected.to contain_keystone_endpoint('RegionOne/nova_service_v3') } - it { is_expected.to contain_keystone_endpoint('RegionOne/nova_service_ec2') } + it { is_expected.to contain_keystone_service('nova_service::compute') } + it { is_expected.to contain_keystone_service('nova_service_v3::computev3') } + it { is_expected.to contain_keystone_service('nova_service_ec2::ec2') } + it { is_expected.to contain_keystone_endpoint('RegionOne/nova_service::compute') } + it { is_expected.to contain_keystone_endpoint('RegionOne/nova_service_v3::computev3') } + it { is_expected.to contain_keystone_endpoint('RegionOne/nova_service_ec2::ec2') } end diff --git a/openstacklib/lib/puppet/parser/functions/os_database_connection.rb b/openstacklib/lib/puppet/parser/functions/os_database_connection.rb index 8764f1b8f..4cc428d94 100644 --- a/openstacklib/lib/puppet/parser/functions/os_database_connection.rb +++ b/openstacklib/lib/puppet/parser/functions/os_database_connection.rb @@ -22,8 +22,9 @@ end v.keys.each do |key| - unless (v[key].class == String) or (v[key] == :undef) - raise(Puppet::ParseError, "os_database_connection(): #{key} should be a String") + klass = (key == 'extra') ? Hash : String + unless (v[key].class == klass) or (v[key] == :undef) + raise(Puppet::ParseError, "os_database_connection(): #{key} should be a #{klass}") end end @@ -56,10 +57,18 @@ end end + # support previous charset option on the function. Setting charset will + # override charset if passed in via the extra parameters if v.include?('charset') - parts[:query] = "charset=#{v['charset']}" + if v.include?('extra') + v['extra'].merge!({ 'charset' => v['charset'] }) + else + v['extra'] = { 'charset' => v['charset'] } + end end + parts[:query] = v['extra'].map{ |k,v| "#{k}=#{v}" }.join('&') if v.include?('extra') + parts[:scheme] = v['dialect'] if v.include?('host') diff --git a/openstacklib/spec/functions/os_database_connection_spec.rb b/openstacklib/spec/functions/os_database_connection_spec.rb index 62d03f21a..f45db924b 100644 --- a/openstacklib/spec/functions/os_database_connection_spec.rb +++ b/openstacklib/spec/functions/os_database_connection_spec.rb @@ -22,12 +22,22 @@ and_raise_error(Puppet::ParseError, /Wrong number of arguments/) end + it 'refuses extra params passed as String' do + is_expected.to run.with_params({ + 'dialect' => 'sqlite', + 'database' => '/var/lib/keystone/keystone.db', + 'host' => '127.0.0.1', + 'port' => '3306', + 'extra' => 'charset=utf-8' + }).and_raise_error(Puppet::ParseError, /extra should be a Hash/) + end + it 'fails if port is provided with missing host' do is_expected.to run.with_params({ 'dialect' => 'sqlite', 'database' => '/var/lib/keystone/keystone.db', 'port' => '3306', - 'charset' => 'utf-8' + 'extra' => { 'charset' => 'utf-8' } }).and_raise_error(Puppet::ParseError, /host is required with port/) end @@ -41,8 +51,21 @@ 'database' => 'test', 'username' => 'guest', 'password' => 's3cr3t', - 'charset' => 'utf-8' - }).and_return('mysql://guest:s3cr3t@127.0.0.1:3306/test?charset=utf-8') + 'extra' => { 'charset' => 'utf-8', 'read_timeout' => '60' } + }).and_return('mysql://guest:s3cr3t@127.0.0.1:3306/test?charset=utf-8&read_timeout=60') + end + + it 'with all parameters and charset set' do + is_expected.to run.with_params({ + 'dialect' => 'mysql', + 'host' => '127.0.0.1', + 'port' => '3306', + 'database' => 'test', + 'username' => 'guest', + 'password' => 's3cr3t', + 'charset' => 'utf-8', + 'extra' => { 'charset' => 'latin1', 'read_timeout' => '60' } + }).and_return('mysql://guest:s3cr3t@127.0.0.1:3306/test?charset=utf-8&read_timeout=60') end it 'without port' do @@ -52,7 +75,7 @@ 'database' => 'test', 'username' => 'guest', 'password' => 's3cr3t', - 'charset' => 'utf-8' + 'extra' => { 'charset' => 'utf-8' } }).and_return('mysql://guest:s3cr3t@127.0.0.1/test?charset=utf-8') end @@ -60,7 +83,7 @@ is_expected.to run.with_params({ 'dialect' => 'sqlite', 'database' => '/var/lib/keystone/keystone.db', - 'charset' => 'utf-8' + 'extra' => { 'charset' => 'utf-8' } }).and_return('sqlite:////var/lib/keystone/keystone.db?charset=utf-8') end @@ -70,7 +93,7 @@ 'host' => '127.0.0.1', 'port' => '3306', 'database' => 'test', - 'charset' => 'utf-8' + 'extra' => { 'charset' => 'utf-8' } }).and_return('mysql://127.0.0.1:3306/test?charset=utf-8') end @@ -81,7 +104,7 @@ 'port' => '3306', 'database' => 'test', 'username' => :undef, - 'charset' => 'utf-8' + 'extra' => { 'charset' => 'utf-8' } }).and_return('mysql://127.0.0.1:3306/test?charset=utf-8') end @@ -92,7 +115,7 @@ 'port' => '3306', 'database' => 'test', 'username' => '', - 'charset' => 'utf-8' + 'extra' => { 'charset' => 'utf-8' } }).and_return('mysql://127.0.0.1:3306/test?charset=utf-8') end @@ -103,7 +126,7 @@ 'port' => '3306', 'database' => 'test', 'username' => 'guest', - 'charset' => 'utf-8' + 'extra' => { 'charset' => 'utf-8' } }).and_return('mysql://guest@127.0.0.1:3306/test?charset=utf-8') end @@ -115,7 +138,7 @@ 'database' => 'test', 'username' => 'guest', 'password' => :undef, - 'charset' => 'utf-8' + 'extra' => { 'charset' => 'utf-8' } }).and_return('mysql://guest@127.0.0.1:3306/test?charset=utf-8') end @@ -127,7 +150,7 @@ 'database' => 'test', 'username' => 'guest', 'password' => '', - 'charset' => 'utf-8' + 'extra' => { 'charset' => 'utf-8' } }).and_return('mysql://guest@127.0.0.1:3306/test?charset=utf-8') end end diff --git a/sahara/spec/classes/sahara_keystone_auth_spec.rb b/sahara/spec/classes/sahara_keystone_auth_spec.rb index a22bc3a9a..1eaf5a9ea 100644 --- a/sahara/spec/classes/sahara_keystone_auth_spec.rb +++ b/sahara/spec/classes/sahara_keystone_auth_spec.rb @@ -25,13 +25,12 @@ :roles => ['admin'] )} - it { is_expected.to contain_keystone_service('sahara').with( + it { is_expected.to contain_keystone_service('sahara::data-processing').with( :ensure => 'present', - :type => 'data-processing', :description => 'Sahara Data Processing' ) } - it { is_expected.to contain_keystone_endpoint('RegionOne/sahara').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/sahara::data-processing').with( :ensure => 'present', :public_url => "http://127.0.0.1:8386/v1.1/%(tenant_id)s", :admin_url => "http://127.0.0.1:8386/v1.1/%(tenant_id)s", @@ -58,7 +57,7 @@ :admin_url => 'http://10.10.10.12:81/v1.1/%(tenant_id)s' } end - it { is_expected.to contain_keystone_endpoint('RegionOne/sahara').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/sahara::data-processing').with( :ensure => 'present', :public_url => 'https://10.10.10.10:80/v1.1/%(tenant_id)s', :internal_url => 'http://10.10.10.11:81/v1.1/%(tenant_id)s', @@ -74,7 +73,7 @@ it { is_expected.to contain_keystone_user('saharay') } it { is_expected.to contain_keystone_user_role('saharay@services') } - it { is_expected.to contain_keystone_service('saharay') } - it { is_expected.to contain_keystone_endpoint('RegionOne/saharay') } + it { is_expected.to contain_keystone_service('saharay::data-processing') } + it { is_expected.to contain_keystone_endpoint('RegionOne/saharay::data-processing') } end end diff --git a/swift/README.md b/swift/README.md index e53475953..d0297155b 100644 --- a/swift/README.md +++ b/swift/README.md @@ -22,7 +22,7 @@ The swift module is a part of [OpenStack](https://github.com/openstack), an effo Module Description ------------------ -The swift module is a thorough attempt to make Puppet capable of managing the entirety of swift. This includes manifests to provision such things as keystone, storage backends, proxies, and the ring. Types are shipped as part of the swift module to assist in manipulation of configuration files. The classes in this module will deploy Swift using best practices for a typical deployment. +The swift module is a thorough attempt to make Puppet capable of managing the entirety of swift. This includes manifests to provision such things as keystone, storage backends, proxies, and the ring. Types are shipped as part of the swift module to assist in manipulation of configuration files. A custom service provider built around the swift-init tool is also provided as an option for enhanced swift service management. The classes in this module will deploy Swift using best practices for a typical deployment. This module is tested in combination with other modules needed to build and leverage an entire Openstack software stack. These modules can be found, all pulled together in the [openstack module](https://github.com/stackforge/puppet-openstack). In addition, this module requires Puppet's [exported resources](http://docs.puppetlabs.com/puppet/3/reference/lang_exported.html). @@ -88,8 +88,7 @@ class { 'swift::storage::all': weight => 1, } -Swift::Ringsync<<||>> -``` +Swift::Ringsync<<||>>``` Usage ----- @@ -245,6 +244,85 @@ class { 'swift::objectexpirer': } It is assumed that the object expirer service will usually be installed in a proxy node. On Red Hat-based distributions, if the class is included in a non-proxy node, the openstack-swift-proxy package will need to be installed. + +##Swiftinit service provider + +The 'swiftinit' provider is a custom provider of the service type. + +"Swift services are generally managed with swift-init. the general usage is swift-init , where service is the swift service to manage (for example object, container, account, proxy)" +From http://docs.openstack.org/developer/swift/admin_guide.html#managing-services + +This new provider is intended to improve puppet-swift deployments in the following ways: + +* The default service provider for puppet-swift is to use distribution specific service providers such as systemd and upstart. If distribution provided init scripts do not specify the full range of service commands, puppet will fall back to methods such as process name matching which is not very reliable. For example, if you were to tail a log file with the same name as a swift process, puppet will interpret that process table match as the swift-proxy service running and fail to start the swift service. +* Minimize customer impact: Using the swiftinit service provider enables more specific and targeted control of swift services. Swift-init provides grateful stop/start reload/restart of swift services which will allow swift processes to finish any current requests before completely stopping the old processes. +* Specific control of services starting at boot is implemented by adding or removing +a templated init or services file. This is managed by this provider. For EL and non Ubuntu Debian OS types, this provider will also make calls out to systemctl reload and systemctl enable/disable. +* Future use of the swiftinit provider is planed to allow for starting multiple servers using swift-init and multiple configuration files, to support a dedicated replication network. + + +### Using the swiftinit service provider +* To use the swiftinit service provider set "service_provider" on the supported components you have defined in your config manifest. + +``` + class { '::swift::storage::account': + service_provider => 'swiftinit', + } + class { '::swift::storage::container': + service_provider => 'swiftinit', + } + class { '::swift::storage::object': + service_provider => 'swiftinit', + } + class {'::swift::objectexpirer': + service_provider => 'swiftinit', + } + class { '::swift::proxy': + service_provider => 'swiftinit', + } +``` + +Moving from the default service providers to the swiftinit service provider is supported. On the next puppet run after setting the swiftinit service provider swift services are stopped on the old provider and immediately started using swift-init. This provides a supported upgrade path with no down time. + +The swiftinit service provider uses the following service type parameters to +manage swift services in a non standard way. + +* `manifest` is used to pass in the config file the service should be +configured with. Ex `object-server.conf` +* `pattern` is used to pass in the debian/redhat osfamily specific service names as found in params.pp. Used to match names on services files as provided by distro packages. Debian/Ubuntu service names already match names used by swift-init. + +To aid with input validation to the swiftinit provider there is a defined type swift::service + +### Class: swift::service + +This is a wrapper defined type for the swift service providers. +It provides a centraziled location to manage and validate in put for use to the default +as well as the swiftinit service providers. + +####`namevar` +The namevar/title of swift::service must be one of the swift_init_service_names listed in swift::params.pp. +These names are parsed by the swiftinit provider to provide service management as well as template boot files. + +####`os_family_service_name` +The distribution specific service name from swift::params. This name is passed to the default service provider. +This name is used by the swiftinit provider to match on default provider service names when moving from a default +provider to the swiftinit provider. The swiftinit provider also uses the service_name to manage service and init files. + +####`config_file_name` +The swift service configuration file name. It must be one of the following: +object-server.conf, account-server.conf, container-server.conf, proxy-server.conf, object-expirer.conf. + +####`service_ensure` +The state of the service to ensure, running or stopped. + +####`enabled` +Should the service be enabled to start at boot. + +####`service_provider` +To use the swiftinit service provider to manage swift services, set service_provider to "swiftinit". When enable is true the provider +will populate boot files that start swift using swift-init at boot. Defaults to $::swift::params::service_provider. + + ### Verifying installation This modules ships with a simple Ruby script that validates whether or not your swift cluster is functional. @@ -258,7 +336,7 @@ Implementation ### swift -swift is a combination of Puppet manifest and ruby code to delivery configuration and extra functionality through types and providers. +puppet-swift is a combination of Puppet manifest and ruby code to deliver configuration and extra functionality through types and providers. ### Types diff --git a/swift/files/swift-account.conf.upstart b/swift/files/swift-account.conf.upstart deleted file mode 100644 index cc85cd73c..000000000 --- a/swift/files/swift-account.conf.upstart +++ /dev/null @@ -1,23 +0,0 @@ -# Temporarily managed by Puppet until -# 931893 is resolved -# swift-account-server - SWIFT Object Server -# -# The swift account server. - -description "SWIFT Account Server" -author "Marc Cluet " - -start on runlevel [2345] -stop on runlevel [016] - -pre-start script - if [ $(find /etc/swift/account-server/ -type f 2>/dev/null | wc -l) -gt 0 ]; then - exec /usr/bin/swift-init account-server start - elif [ -f /etc/swift/account-server.conf ]; then - exec /usr/bin/swift-init account-server start - else - exit 1 - fi -end script - -post-stop exec /usr/bin/swift-init account-server stop diff --git a/swift/files/swift-container-sync.conf.upstart b/swift/files/swift-container-sync.conf.upstart deleted file mode 100644 index 54b010cf7..000000000 --- a/swift/files/swift-container-sync.conf.upstart +++ /dev/null @@ -1,20 +0,0 @@ -# swift-container-sync - SWIFT Container Sync -# -# The swift container sync. - -description "SWIFT Container Sync" -author "Sergio Rubio " - -start on runlevel [2345] -stop on runlevel [016] - -pre-start script - if [ -f "/etc/swift/container-server.conf" ]; then - exec /usr/bin/swift-init container-sync start - else - exit 1 - fi -end script - -post-stop exec /usr/bin/swift-init container-sync stop - diff --git a/swift/files/swift-container.conf.upstart b/swift/files/swift-container.conf.upstart deleted file mode 100644 index aeaab6511..000000000 --- a/swift/files/swift-container.conf.upstart +++ /dev/null @@ -1,23 +0,0 @@ -# Temporarily managed by Puppet until -# 931893 is resolved -# swift-container-server - SWIFT Object Server -# -# The swift container server. - -description "SWIFT Container Server" -author "Marc Cluet " - -start on runlevel [2345] -stop on runlevel [016] - -pre-start script - if [ $(find /etc/swift/container-server/ -type f 2>/dev/null | wc -l) -gt 0 ]; then - exec /usr/bin/swift-init container-server start - elif [ -f /etc/swift/container-server.conf ]; then - exec /usr/bin/swift-init container-server start - else - exit 1 - fi -end script - -post-stop exec /usr/bin/swift-init container-server stop diff --git a/swift/files/swift-object.conf.upstart b/swift/files/swift-object.conf.upstart deleted file mode 100644 index 45eb2f773..000000000 --- a/swift/files/swift-object.conf.upstart +++ /dev/null @@ -1,23 +0,0 @@ -# Temporarily managed by Puppet until -# 931893 is resolved -# swift-object-server - SWIFT Object Server -# -# The swift object server. - -description "SWIFT Object Server" -author "Marc Cluet " - -start on runlevel [2345] -stop on runlevel [016] - -pre-start script - if [ $(find /etc/swift/object-server/ -type f 2>/dev/null | wc -l) -gt 0 ]; then - exec /usr/bin/swift-init object-server start - elif [ -f /etc/swift/object-server.conf ]; then - exec /usr/bin/swift-init object-server start - else - exit 1 - fi -end script - -post-stop exec /usr/bin/swift-init object-server stop diff --git a/swift/lib/puppet/provider/service/swiftinit.rb b/swift/lib/puppet/provider/service/swiftinit.rb new file mode 100644 index 000000000..e46a315cf --- /dev/null +++ b/swift/lib/puppet/provider/service/swiftinit.rb @@ -0,0 +1,210 @@ +# swift-init service management +# +# author Adam Vinsh +Puppet::Type.type(:service).provide :swiftinit, :parent => :service do + desc 'Manage swift services using swift-init' + + has_feature :enableable + has_feature :refreshable + + confine :any => [ + Facter.value(:osfamily) == 'Debian', + Facter.value(:osfamily) == 'RedHat' + ] + + # Check if swift service is running using swift-init + def status + if swiftinit_run('status', false).exitstatus == 0 + return :running + else + # Transition block for systemd systems. If swift-init reports service is + # not running then send stop to systemctl so that service can be started + # with swift-init and fully managed by this provider. + if Facter.value(:operatingsystem) != 'Ubuntu' + systemctl_run('stop', [resource[:pattern]], false) + systemctl_run('disable', [resource[:pattern]], false) + end + return :stopped + end + end + + # Start this swift service using swift-init + def start + swiftinit_run('start', true) + end + + # Stop this swift service using swift-init allowing + # current requests to finish on supporting servers + def stop + swiftinit_run('shutdown', true) + end + + # Restart this swift service using swift-init reload, + # graceful shutdown then restart on supporting servers + def restart + swiftinit_run('reload', true) + end + + def refresh + if (@paramaters[:ensure] == running) + provider.restart + else + debug 'Skipping restart, service is not running' + end + end + + # Returns service enabled status first checking for init/systemd file + # presence then checking if file content matches this provider and not + # distro provided. Also on Redhat/Debian checks systemctl status. + def enabled? + if Facter.value(:operatingsystem) != 'Ubuntu' + if Puppet::FileSystem.exist?("/etc/systemd/system/#{resource[:pattern]}.service") + current_conf = File.read("/etc/systemd/system/#{resource[:pattern]}.service") + if !current_conf.eql? systemd_template + return :false + end + if systemctl_run('is-enabled', [resource[:pattern]], false).exitstatus == 0 + return :true + end + else + return :false + end + elsif Facter.value(:operatingsystem) == 'Ubuntu' + if Puppet::FileSystem.exist?("/etc/init/#{resource[:pattern]}.conf") + current_conf = File.read("/etc/init/#{resource[:pattern]}.conf") + if current_conf.eql? upstart_template + return :true + end + else + return :false + end + end + end + + # Enable the service at boot. For Redhat and Debian create services + # file and notify systemctl. For Ubuntu create init file. + def enable + if Facter.value(:operatingsystem) != 'Ubuntu' + file = Puppet::Type.type(:file).new( + :name => "/etc/systemd/system/#{resource[:pattern]}.service", + :ensure => :present, + :content => systemd_template, + :mode => '0644' + ) + file.write(file) + systemctl_run('daemon-reload', nil, true) + systemctl_run('enable', [resource[:pattern]], false) + elsif Facter.value(:operatingsystem) == 'Ubuntu' + file = Puppet::Type.type(:file).new( + :name => "/etc/init/#{resource[:pattern]}.conf", + :ensure => :present, + :content => upstart_template, + :mode => '0644' + ) + file.write(file) + end + end + + # Disable the service at boot. For Redhat and Debain, + # delete services file and notify systemctl. For Ubuntu + # remove init file. + def disable + if Facter.value(:operatingsystem) != 'Ubuntu' + systemctl_run('disable', [resource[:pattern]], false) + File.delete("/etc/systemd/system/#{resource[:pattern]}.service") + systemctl_run('daemon-reload', nil, true) + elsif Facter.value(:operatingsystem) == 'Ubuntu' + File.delete("/etc/init/#{resource[:pattern]}.conf") + end + end + + # Wrapper to handle swift-init calls on supported osfamily + def swiftinit_run(command, failonfail) + execute([['swift-init'], ["#{type}-#{subtype}#{manifest}"], [command]], + :failonfail => failonfail) + rescue Puppet::ExecutionFailure => detail + @resource.fail Puppet::Error, + "swift-init #{type}-#{subtype}#{manifest} #{command} + failed with: #{@resource.ref}: #{detail}", detail + end + + # Wrapper to handle systemctl calls on supported osfamily + def systemctl_run(command, unit_file, failonfail) + if unit_file + execute([['systemctl'], [command], [unit_file]], :failonfail => failonfail) + else + execute([['systemctl'], [command]], :failonfail => failonfail) + end + rescue Puppet::ExecutionFailure => detail + @resource.fail Puppet::Error, + "systemctl #{command} #{unit_file} + failed with: #{@resource.ref}: #{detail}", detail + end + + # Split the service type off of name + # type can be object, account, container. + def type + resource[:name].split(/-/)[1] + end + + # Split the service subtype off of name + # subtype can be: + # For type account: auditor, reaper, replicator, server. + # For type container: auditor, replicator, server, sync, updater. + # For type object: auditor, replicator, server, updater, expirer. + # For type proxy: server. + def subtype + resource[:name].split(/-/)[2] + end + + # In this provider 'manifest' is the name of the config file that the service + # uses to run. If the config file is a default name ex: object-server.conf. + # then swift-init can be called without specifying the config file. + # TODO add logic to start servers using multiple config files, used to run + # swift with a dedicated replication network. + def manifest + if "#{resource[:manifest]}" == "#{type}-server.conf" + return nil + elsif "#{resource[:manifest]}" == 'object-expirer.conf' + return nil + else return ".#{resource[:manifest].split('.conf')[1]}" + end + end + + # Begin service template boot section. + def upstart_template + %(# swift-#{type}-#{subtype} +# +# Starts the swift-#{type}-#{subtype}. + +description "SWIFT #{type} #{subtype}" +author "Puppet" + +start on runlevel [2345] +stop on runlevel [016] + +pre-start script +if [ -f /etc/swift/#{resource[:manifest]} ]; then + exec /usr/bin/swift-init #{type}-#{subtype} start +else + exit 1 +fi +end script + +post-stop exec /usr/bin/swift-init #{type}-#{subtype} stop) + end + + def systemd_template + %([Unit] +Description=OpenStack "SWIFT #{type} #{subtype}" +After=syslog.target network.target + +[Service] +Type=forking +User=root +ExecStart=/usr/bin/swift-init #{type}-#{subtype} start + +[Install] +WantedBy=multi-user.target) + end +end diff --git a/swift/lib/puppet/provider/swift_ring_builder.rb b/swift/lib/puppet/provider/swift_ring_builder.rb index 2b30cc8bd..a34f2e69f 100644 --- a/swift/lib/puppet/provider/swift_ring_builder.rb +++ b/swift/lib/puppet/provider/swift_ring_builder.rb @@ -20,19 +20,41 @@ def self.address_string(address) def self.lookup_ring object_hash = {} if File.exists?(builder_file_path) + # Swift < 2.2.2 Skip first 4 info lines from swift-ring-builder output if rows = swift_ring_builder(builder_file_path).split("\n")[4..-1] + # Swift 2.2.2+ Skip additional line to account for Overload info + if !rows[0].nil? and rows[0].start_with?('Devices:') + rows.shift + end rows.each do |row| # Swift 1.7+ output example: + # /etc/swift/object.builder, build version 1 + # 262144 partitions, 1.000000 replicas, 1 regions, 1 zones, 1 devices, 0.00 balance, 0.00 dispersion + # The minimum number of hours before a partition can be reassigned is 1 # Devices: id region zone ip address port name weight partitions balance meta # 0 1 2 127.0.0.1 6022 2 1.00 262144 0.00 # 0 1 3 192.168.101.15 6002 1 1.00 262144 -100.00 # # Swift 1.8.0 output example: + # /etc/swift/object.builder, build version 1 + # 262144 partitions, 1.000000 replicas, 1 regions, 1 zones, 1 devices, 0.00 balance, 0.00 dispersion + # The minimum number of hours before a partition can be reassigned is 1 # Devices: id region zone ip address port name weight partitions balance meta # 2 1 2 192.168.101.14 6002 1 1.00 262144 200.00 m2 # 0 1 3 192.168.101.15 6002 1 1.00 262144-100.00 m2 # # Swift 1.8+ output example: + # /etc/swift/object.builder, build version 1 + # 262144 partitions, 1.000000 replicas, 1 regions, 1 zones, 1 devices, 0.00 balance, 0.00 dispersion + # The minimum number of hours before a partition can be reassigned is 1 + # Devices: id region zone ip address port replication ip replication port name weight partitions balance meta + # 0 1 2 127.0.0.1 6021 127.0.0.1 6021 2 1.00 262144 0.00 + # + # Swift 2.2.2+ output example: + # /etc/swift/object.builder, build version 1 + # 262144 partitions, 1.000000 replicas, 1 regions, 1 zones, 1 devices, 0.00 balance, 0.00 dispersion + # The minimum number of hours before a partition can be reassigned is 1 + # The overload factor is 0.00% (0.000000) # Devices: id region zone ip address port replication ip replication port name weight partitions balance meta # 0 1 2 127.0.0.1 6021 127.0.0.1 6021 2 1.00 262144 0.00 # Swift 1.8+ output example: diff --git a/swift/manifests/init.pp b/swift/manifests/init.pp index a7ebb135a..3098eb33e 100644 --- a/swift/manifests/init.pp +++ b/swift/manifests/init.pp @@ -55,7 +55,8 @@ ensure => directory, } user {'swift': - ensure => present, + ensure => present, + require => Package['swift'], } file { '/var/lib/swift': ensure => directory, diff --git a/swift/manifests/objectexpirer.pp b/swift/manifests/objectexpirer.pp index ecf0dcd52..0cd3d75d7 100644 --- a/swift/manifests/objectexpirer.pp +++ b/swift/manifests/objectexpirer.pp @@ -56,7 +56,14 @@ # (optional) Report interval, in seconds. # Defaults to 300. # - +# [*service_provider*] +# (optional) +# To use the swiftinit service provider to manage swift services, set +# service_provider to "swiftinit". When enable is true the provider +# will populate boot files that start swift using swift-init at boot. +# See README for more details. +# Defaults to $::swift::params::service_provider. +# class swift::objectexpirer( $manage_service = true, $enabled = true, @@ -71,9 +78,8 @@ $reclaim_age = 604800, $recon_cache_path = '/var/cache/swift', $report_interval = 300, -) { - - include ::swift::params + $service_provider = $::swift::params::service_provider +) inherits ::swift::params { Swift_config<| |> ~> Service['swift-object-expirer'] Swift_object_expirer_config<||> ~> Service['swift-object-expirer'] @@ -109,12 +115,11 @@ } } - service { 'swift-object-expirer': - ensure => $service_ensure, - name => $::swift::params::object_expirer_service_name, - enable => $enabled, - provider => $::swift::params::service_provider, - tag => 'swift-service', + swift::service { 'swift-object-expirer': + os_family_service_name => $::swift::params::object_expirer_service_name, + service_ensure => $service_ensure, + enabled => $enabled, + config_file_name => 'object-expirer.conf', + service_provider => $service_provider, } } - diff --git a/swift/manifests/params.pp b/swift/manifests/params.pp index a4aa65d89..dc82cf283 100644 --- a/swift/manifests/params.pp +++ b/swift/manifests/params.pp @@ -6,21 +6,22 @@ $package_name = 'swift' $client_package = 'python-swiftclient' $proxy_package_name = 'swift-proxy' - $proxy_service_name = 'swift-proxy' + $proxy_server_service_name = 'swift-proxy' $object_package_name = 'swift-object' - $object_service_name = 'swift-object' + $object_server_service_name = 'swift-object' $object_auditor_service_name = 'swift-object-auditor' $object_replicator_service_name = 'swift-object-replicator' $object_updater_service_name = 'swift-object-updater' $object_expirer_package_name = 'swift-object-expirer' $object_expirer_service_name = 'swift-object-expirer' $container_package_name = 'swift-container' - $container_service_name = 'swift-container' + $container_server_service_name = 'swift-container' $container_auditor_service_name = 'swift-container-auditor' $container_replicator_service_name = 'swift-container-replicator' $container_updater_service_name = 'swift-container-updater' + $container_sync_service_name = 'swift-container-sync' $account_package_name = 'swift-account' - $account_service_name = 'swift-account' + $account_server_service_name = 'swift-account' $account_auditor_service_name = 'swift-account-auditor' $account_reaper_service_name = 'swift-account-reaper' $account_replicator_service_name = 'swift-account-replicator' @@ -35,21 +36,22 @@ $package_name = 'openstack-swift' $client_package = 'python-swiftclient' $proxy_package_name = 'openstack-swift-proxy' - $proxy_service_name = 'openstack-swift-proxy' + $proxy_server_service_name = 'openstack-swift-proxy' $object_package_name = 'openstack-swift-object' - $object_service_name = 'openstack-swift-object' + $object_server_service_name = 'openstack-swift-object' $object_auditor_service_name = 'openstack-swift-object-auditor' $object_replicator_service_name = 'openstack-swift-object-replicator' $object_updater_service_name = 'openstack-swift-object-updater' $object_expirer_package_name = 'openstack-swift-proxy' $object_expirer_service_name = 'openstack-swift-object-expirer' $container_package_name = 'openstack-swift-container' - $container_service_name = 'openstack-swift-container' + $container_server_service_name = 'openstack-swift-container' $container_auditor_service_name = 'openstack-swift-container-auditor' $container_replicator_service_name = 'openstack-swift-container-replicator' $container_updater_service_name = 'openstack-swift-container-updater' + $container_sync_service_name = 'openstack-swift-container-sync' $account_package_name = 'openstack-swift-account' - $account_service_name = 'openstack-swift-account' + $account_server_service_name = 'openstack-swift-account' $account_auditor_service_name = 'openstack-swift-account-auditor' $account_reaper_service_name = 'openstack-swift-account-reaper' $account_replicator_service_name = 'openstack-swift-account-replicator' @@ -60,4 +62,21 @@ fail("Unsupported osfamily: ${::osfamily} for os ${::operatingsystem}") } } + $swift_init_service_names = [ + 'swift-proxy-server', + 'swift-object-auditor', + 'swift-object-expirer', + 'swift-object-replicator', + 'swift-object-server', + 'swift-object-updater', + 'swift-account-auditor', + 'swift-account-reaper', + 'swift-account-replicator', + 'swift-account-server', + 'swift-container-auditor', + 'swift-container-replicator', + 'swift-container-server', + 'swift-container-sync', + 'swift-container-updater', + ] } diff --git a/swift/manifests/proxy.pp b/swift/manifests/proxy.pp index bec925e2a..7a9f898b0 100644 --- a/swift/manifests/proxy.pp +++ b/swift/manifests/proxy.pp @@ -98,6 +98,14 @@ # Configures log_name for swift proxy-server. # Optional. Defaults to proxy-server # +# [*service_provider*] +# (optional) +# To use the swiftinit service provider to manage swift services, set +# service_provider to "swiftinit". When enable is true the provider +# will populate boot files that start swift using swift-init at boot. +# See README for more details. +# Defaults to $::swift::params::service_provider. +# # == Examples # # == Authors @@ -129,13 +137,13 @@ $node_timeout = undef, $manage_service = true, $enabled = true, - $package_ensure = 'present' -) { + $package_ensure = 'present', + $service_provider = $::swift::params::service_provider +) inherits ::swift::params { - include ::swift::params include ::concat::setup - Swift_config<| |> ~> Service['swift-proxy'] + Swift_config<| |> ~> Service['swift-proxy-server'] validate_bool($account_autocreate) validate_bool($allow_account_management) @@ -209,13 +217,12 @@ } } - service { 'swift-proxy': - ensure => $service_ensure, - name => $::swift::params::proxy_service_name, - enable => $enabled, - provider => $::swift::params::service_provider, - hasstatus => true, - subscribe => Concat['/etc/swift/proxy-server.conf'], - tag => 'swift-service', + swift::service { 'swift-proxy-server': + os_family_service_name => $::swift::params::proxy_server_service_name, + service_ensure => $service_ensure, + enabled => $enabled, + config_file_name => 'proxy-server.conf', + service_provider => $service_provider, + subscribe => Concat['/etc/swift/proxy-server.conf'], } } diff --git a/swift/manifests/proxy/ceilometer.pp b/swift/manifests/proxy/ceilometer.pp index a829e8b28..bdcba47d7 100644 --- a/swift/manifests/proxy/ceilometer.pp +++ b/swift/manifests/proxy/ceilometer.pp @@ -7,6 +7,43 @@ # # == Parameters # +# [*rabbit_host*] +# (Optional) IP or hostname of the rabbit server. +# Defaults to '127.0.0.1'. +# +# [*rabbit_port*] +# (Optional) Port of the rabbit server. +# Defaults to 5672. +# +# [*rabbit_hosts*] +# (Optional) IP or hostname of the rabbits servers. +# comma separated array (ex: ['1.0.0.10:5672','1.0.0.11:5672']) +# Defaults to undef. +# +# [*rabbit_user*] +# (Optional) Username for rabbit. +# Defaults to 'guest'. +# +# [*rabbit_password*] +# (Optional) Password for rabbit user. +# Defaults to 'guest'. +# +# [*rabbit_virtual_host*] +# (Optional) Virtual host to use. +# Defaults to '/'. +# +# [*driver*] +# (Optional) The Drivers(s) to handle sending notifications. +# Defaults to undef. +# +# [*topic*] +# (Optional) AMQP topic used for OpenStack notifications. +# Defaults to undef. +# +# [*control_exchange*] +# (Optional) The default exchange under which topics are scoped. +# Defaults to undef. +# # [*ensure*] # Enable or not ceilometer fragment # Defaults to 'present' @@ -28,8 +65,17 @@ # Copyright 2013 eNovance licensing@enovance.com # class swift::proxy::ceilometer( - $ensure = 'present', - $group = 'ceilometer', + $rabbit_user = 'guest', + $rabbit_password = 'guest', + $rabbit_host = '127.0.0.1', + $rabbit_port = '5672', + $rabbit_hosts = undef, + $rabbit_virtual_host = '/', + $driver = undef, + $topic = undef, + $control_exchange = undef, + $ensure = 'present', + $group = 'ceilometer', ) inherits swift { User['swift'] { @@ -54,4 +100,9 @@ require => Class['::ceilometer'], } + package { 'python-ceilometermiddleware': + ensure => $ensure, + tag => 'openstack', + } + } diff --git a/swift/manifests/service.pp b/swift/manifests/service.pp new file mode 100644 index 000000000..4682c4b91 --- /dev/null +++ b/swift/manifests/service.pp @@ -0,0 +1,71 @@ +# == Define: swift::service +# +# Wrapper class to configure swift service providers +# +# === Parameters: +# +# [*title*] The name of the swift service to manage +# Mandatory. ex: 'swift-object-server' valid names +# are listed in swift::params.pp at $swift_init_service_names +# +# [*os_family_service_name*] +# (required) The distribution specific service name. +# +# [*config_file_name*] +# (required) The service configuration file name. +# Starting at the path "/etc/swift/" ex:"object-server.conf" +# +# [*service_ensure*] +# (optional) State of service to ensure, running or stopped. +# Default: undef +# +# [*enabled*] +# (optional) Should the service be enabled to start +# at boot. Default: true +# +# [*service_provider*] +# (optional) +# To use the swiftinit service provider to manage swift services, set +# service_provider to "swiftinit". When enable is true the provider +# will populate boot files that start swift using swift-init at boot. +# See README for more details. +# Defaults to $::swift::params::service_provider. +# +define swift::service( + $os_family_service_name, + $config_file_name, + $service_ensure = undef, + $enabled = true, + $service_provider = $::swift::params::service_provider, +) { + + include ::swift::params + + if(! member($::swift::params::swift_init_service_names, $name)) { + fail("swift::service name: ${name} is not a valid swift_init_service_name") + } + + if $service_provider != 'swiftinit' { + service { $name: + ensure => $service_ensure, + name => $os_family_service_name, + hasstatus => true, + enable => $enabled, + provider => $service_provider, + tag => 'swift-service', + subscribe => $subscribe, + } + } elsif $service_provider == 'swiftinit' { + service { $name: + ensure => $service_ensure, + enable => $enabled, + hasstatus => true, + hasrestart => true, + provider => 'swiftinit', + pattern => $os_family_service_name, + manifest => $config_file_name, + tag => 'swift-service', + subscribe => $subscribe, + } + } +} diff --git a/swift/manifests/storage/account.pp b/swift/manifests/storage/account.pp index 6ea122465..c21bb977a 100644 --- a/swift/manifests/storage/account.pp +++ b/swift/manifests/storage/account.pp @@ -2,8 +2,8 @@ # # == Parameters # [*enabled*] -# (optional) Should the service be enabled. -# Defaults to true +# (optional) Should the service be enabled to start +# at boot. Defaults to true # # [*manage_service*] # (optional) Whether the service should be managed by Puppet. @@ -13,23 +13,37 @@ # (optional) Value of package resource parameter 'ensure'. # Defaults to 'present'. # +# [*config_file_name*] +# (optional) The configuration file name. +# Starting at the path "/etc/swift/" +# Defaults to "account-server.conf" +# +# [*service_provider*] +# (optional) +# To use the swiftinit service provider to manage swift services, set +# service_provider to "swiftinit". When enable is true the provider +# will populate boot files that start swift using swift-init at boot. +# See README for more details. +# Defaults to $::swift::params::service_provider. +# class swift::storage::account( - $manage_service = true, - $enabled = true, - $package_ensure = 'present' -) { + $manage_service = true, + $enabled = true, + $package_ensure = 'present', + $config_file_name = 'account-server.conf', + $service_provider = $::swift::params::service_provider +) inherits ::swift::params { Swift_config<| |> ~> Service['swift-account-reaper'] - Swift_config<| |> ~> Service['swift-account-auditor'] swift::storage::generic { 'account': - manage_service => $manage_service, - enabled => $enabled, - package_ensure => $package_ensure, + manage_service => $manage_service, + enabled => $enabled, + package_ensure => $package_ensure, + config_file_name => $config_file_name, + service_provider => $service_provider } - include ::swift::params - if $manage_service { if $enabled { $service_ensure = 'running' @@ -38,21 +52,12 @@ } } - service { 'swift-account-reaper': - ensure => $service_ensure, - name => $::swift::params::account_reaper_service_name, - enable => $enabled, - provider => $::swift::params::service_provider, - require => Package['swift-account'], - tag => 'swift-service', - } - - service { 'swift-account-auditor': - ensure => $service_ensure, - name => $::swift::params::account_auditor_service_name, - enable => $enabled, - provider => $::swift::params::service_provider, - require => Package['swift-account'], - tag => 'swift-service', + swift::service { 'swift-account-reaper': + os_family_service_name => $::swift::params::account_reaper_service_name, + service_ensure => $service_ensure, + enabled => $enabled, + config_file_name => $config_file_name, + service_provider => $service_provider, + require => Package['swift-account'], } } diff --git a/swift/manifests/storage/container.pp b/swift/manifests/storage/container.pp index f6dc76ab3..067bcb3f9 100644 --- a/swift/manifests/storage/container.pp +++ b/swift/manifests/storage/container.pp @@ -1,9 +1,10 @@ +# Class swift::storage::container # # === Parameters # # [*enabled*] -# (optional) Should the service be enabled. -# Defaults to true +# (optional) Should the service be enabled to start +# at boot. Defaults to true # # [*manage_service*] # (optional) Whether the service should be managed by Puppet. @@ -17,24 +18,38 @@ # (optional) A list of hosts allowed in the X-Container-Sync-To # field for containers. Defaults to one entry list '127.0.0.1'. # +# [*config_file_name*] +# (optional) The configuration file name. +# Starting at the path "/etc/swift/" +# Defaults to "object-server.conf" +# +# [*service_provider*] +# (optional) +# To use the swiftinit service provider to manage swift services, set +# service_provider to "swiftinit". When enable is true the provider +# will populate boot files that start swift using swift-init at boot. +# See README for more details. +# Defaults to $::swift::params::service_provider. +# class swift::storage::container( $manage_service = true, $enabled = true, $package_ensure = 'present', $allowed_sync_hosts = ['127.0.0.1'], -) { + $config_file_name = 'container-server.conf', + $service_provider = $::swift::params::service_provider +) inherits ::swift::params { Swift_config<| |> ~> Service['swift-container-updater'] - Swift_config<| |> ~> Service['swift-container-auditor'] swift::storage::generic { 'container': - manage_service => $manage_service, - enabled => $enabled, - package_ensure => $package_ensure, + manage_service => $manage_service, + enabled => $enabled, + package_ensure => $package_ensure, + config_file_name => $config_file_name, + service_provider => $service_provider } - include ::swift::params - if $manage_service { if $enabled { $service_ensure = 'running' @@ -43,35 +58,23 @@ } } - service { 'swift-container-updater': - ensure => $service_ensure, - name => $::swift::params::container_updater_service_name, - enable => $enabled, - provider => $::swift::params::service_provider, - require => Package['swift-container'], - tag => 'swift-service', + swift::service { 'swift-container-updater': + os_family_service_name => $::swift::params::container_updater_service_name, + service_ensure => $service_ensure, + enabled => $enabled, + config_file_name => $config_file_name, + service_provider => $service_provider, + require => Package['swift-container'], } - service { 'swift-container-auditor': - ensure => $service_ensure, - name => $::swift::params::container_auditor_service_name, - enable => $enabled, - provider => $::swift::params::service_provider, - require => Package['swift-container'], - tag => 'swift-service', - } - - if $::operatingsystem == 'Ubuntu' { - # The following service conf is missing in Ubunty 12.04 - file { '/etc/init/swift-container-sync.conf': - source => 'puppet:///modules/swift/swift-container-sync.conf.upstart', - require => Package['swift-container'], - } - service { 'swift-container-sync': - ensure => $service_ensure, - enable => $enabled, - provider => $::swift::params::service_provider, - require => File['/etc/init/swift-container-sync.conf'], + if $::osfamily == 'Debian' { + swift::service { 'swift-container-sync': + os_family_service_name => $::swift::params::container_sync_service_name, + service_ensure => $service_ensure, + enabled => $enabled, + config_file_name => $config_file_name, + service_provider => $service_provider, + require => Package['swift-container'], } Swift_config<| |> ~> Service['swift-container-sync'] } diff --git a/swift/manifests/storage/generic.pp b/swift/manifests/storage/generic.pp index 763dcc672..7dd849656 100644 --- a/swift/manifests/storage/generic.pp +++ b/swift/manifests/storage/generic.pp @@ -3,8 +3,8 @@ # # == Parameters # [*enabled*] -# (optional) Should the service be enabled. -# Defaults to true +# (optional) Should the service be enabled to start +# at boot. Defaults to true # # [*manage_service*] # (optional) Whether the service should be managed by Puppet. @@ -14,45 +14,51 @@ # (optional) The desired ensure state of the swift storage packages. # Defaults to present. # +# [*config_file_name*] +# (optional) The configuration file name. +# Starting at the path "/etc/swift/" +# Defaults to "${name}-server.conf" +# # [*service_provider*] -# (optional) The provider to use for the service +# (optional) +# To use the swiftinit service provider to manage swift services, set +# service_provider to "swiftinit". When enable is true the provider +# will populate boot files that start swift using swift-init at boot. +# See README for more details. +# Defaults to $::swift::params::service_provider. # # == Dependencies # Requires Class[swift::storage] -# == Examples -# -# == Authors -# -# Dan Bode dan@puppetlabs.com # -# == Copyright -# -# Copyright 2011 Puppetlabs Inc, unless otherwise noted. define swift::storage::generic( $manage_service = true, $enabled = true, $package_ensure = 'present', + $config_file_name = "${name}-server.conf", $service_provider = $::swift::params::service_provider ) { include ::swift::params Class['swift::storage'] -> Swift::Storage::Generic[$name] - Swift_config<| |> ~> Service["swift-${name}"] + Swift_config<| |> ~> Service["swift-${name}-server"] + Swift_config<| |> ~> Service["swift-${name}-auditor"] + Swift_config<| |> ~> Service["swift-${name}-replicator"] validate_re($name, '^object|container|account$') package { "swift-${name}": ensure => $package_ensure, - # this is a way to dynamically build the variables to lookup - # sorry its so ugly :( - name => inline_template("<%= scope.lookupvar('::swift::params::${name}_package_name') %>"), + name => getvar("::swift::params::${name}_package_name"), tag => ['openstack', 'swift-package'], - before => Service["swift-${name}", "swift-${name}-replicator"], + before => Service["swift-${name}-server", "swift-${name}-replicator"], } file { "/etc/swift/${name}-server/": - ensure => directory, + ensure => directory, + owner => 'swift', + group => 'swift', + require => Package["swift-${name}"], } if $manage_service { @@ -63,24 +69,30 @@ } } - service { "swift-${name}": - ensure => $service_ensure, - name => inline_template("<%= scope.lookupvar('::swift::params::${name}_service_name') %>"), - enable => $enabled, - hasstatus => true, - provider => $service_provider, - subscribe => Package["swift-${name}"], - tag => 'swift-service', + swift::service { "swift-${name}-server": + os_family_service_name => getvar("::swift::params::${name}_server_service_name"), + service_ensure => $service_ensure, + enabled => $enabled, + config_file_name => $config_file_name, + service_provider => $service_provider, + subscribe => Package["swift-${name}"], } - service { "swift-${name}-replicator": - ensure => $service_ensure, - name => inline_template("<%= scope.lookupvar('::swift::params::${name}_replicator_service_name') %>"), - enable => $enabled, - hasstatus => true, - provider => $service_provider, - subscribe => Package["swift-${name}"], - tag => 'swift-service', + swift::service { "swift-${name}-replicator": + os_family_service_name => getvar("::swift::params::${name}_replicator_service_name"), + service_ensure => $service_ensure, + enabled => $enabled, + config_file_name => $config_file_name, + service_provider => $service_provider, + subscribe => Package["swift-${name}"], } + swift::service { "swift-${name}-auditor": + os_family_service_name => getvar("::swift::params::${name}_auditor_service_name"), + service_ensure => $service_ensure, + enabled => $enabled, + config_file_name => $config_file_name, + service_provider => $service_provider, + subscribe => Package["swift-${name}"], + } } diff --git a/swift/manifests/storage/object.pp b/swift/manifests/storage/object.pp index 12d044d88..c7e8e43f2 100644 --- a/swift/manifests/storage/object.pp +++ b/swift/manifests/storage/object.pp @@ -2,8 +2,8 @@ # # == Parameters # [*enabled*] -# (optional) Should the service be enabled. -# Defaults to true +# (optional) Should the service be enabled to start +# at boot. Defaults to true # # [*manage_service*] # (optional) Whether the service should be managed by Puppet. @@ -13,23 +13,37 @@ # (optional) Value of package resource parameter 'ensure'. # Defaults to 'present'. # +# [*config_file_name*] +# (optional) The configuration file name. +# Starting at the path "/etc/swift/" +# Defaults to "object-server.conf" +# +# [*service_provider*] +# (optional) +# To use the swiftinit service provider to manage swift services, set +# service_provider to "swiftinit". When enable is true the provider +# will populate boot files that start swift using swift-init at boot. +# See README for more details. +# Defaults to $::swift::params::service_provider. +# class swift::storage::object( - $manage_service = true, - $enabled = true, - $package_ensure = 'present' -) { + $manage_service = true, + $enabled = true, + $package_ensure = 'present', + $config_file_name = 'object-server.conf', + $service_provider = $::swift::params::service_provider +) inherits ::swift::params { Swift_config<| |> ~> Service['swift-object-updater'] - Swift_config<| |> ~> Service['swift-object-auditor'] swift::storage::generic { 'object': - manage_service => $manage_service, - enabled => $enabled, - package_ensure => $package_ensure, + manage_service => $manage_service, + enabled => $enabled, + package_ensure => $package_ensure, + config_file_name => $config_file_name, + service_provider => $service_provider } - include ::swift::params - if $manage_service { if $enabled { $service_ensure = 'running' @@ -38,21 +52,12 @@ } } - service { 'swift-object-updater': - ensure => $service_ensure, - name => $::swift::params::object_updater_service_name, - enable => $enabled, - provider => $::swift::params::service_provider, - require => Package['swift-object'], - tag => 'swift-service', - } - - service { 'swift-object-auditor': - ensure => $service_ensure, - name => $::swift::params::object_auditor_service_name, - enable => $enabled, - provider => $::swift::params::service_provider, - require => Package['swift-object'], - tag => 'swift-service', + swift::service { 'swift-object-updater': + os_family_service_name => $::swift::params::object_updater_service_name, + service_ensure => $service_ensure, + enabled => $enabled, + config_file_name => $config_file_name, + service_provider => $service_provider, + require => Package['swift-object'], } } diff --git a/swift/manifests/storage/server.pp b/swift/manifests/storage/server.pp index d4d8fc87f..fa5f0cfd0 100644 --- a/swift/manifests/storage/server.pp +++ b/swift/manifests/storage/server.pp @@ -40,7 +40,6 @@ # *NOTE*: Recommended parameter: 'Du=rwx,g=rx,o=rx,Fu=rw,g=r,o=r' # This mask translates to 0755 for directories and 0644 for files. # - # [*pipeline*] # (optional) Pipeline of applications. # Defaults to ["${type}-server"]. @@ -93,7 +92,7 @@ # [*log_name*] # (optional) Label used when logging. # Defaults to "${type}-server". - +# # [*log_udp_host*] # (optional) If not set, the UDP receiver for syslog is disabled. # Defaults to undef. @@ -107,9 +106,10 @@ # good for seeing errors if true # Defaults to true. # -# [*config_file_path*] -# (optional) The configuration file name. -# Defaults to "${type}-server/${name}.conf". +# [*config_file_path*] +# (optional) The configuration file name. +# Starting at the path "/etc/swift/" +# Defaults to "${type}-server.conf" # define swift::storage::server( $type, @@ -136,7 +136,7 @@ $log_udp_port = undef, $log_requests = true, # this parameters needs to be specified after type and name - $config_file_path = "${type}-server/${name}.conf" + $config_file_path = "${type}-server.conf", ) { if ($incoming_chmod == '0644') { @@ -169,6 +169,7 @@ } include "::swift::storage::${type}" + include ::concat::setup validate_re($name, '^\d+$') @@ -193,7 +194,7 @@ concat { "/etc/swift/${config_file_path}": owner => $owner, group => $group, - notify => Service["swift-${type}", "swift-${type}-replicator"], + notify => Service["swift-${type}-server", "swift-${type}-replicator"], require => Package['swift'], } diff --git a/swift/spec/acceptance/basic_swift_spec.rb b/swift/spec/acceptance/basic_swift_spec.rb index acfee473b..50212b762 100644 --- a/swift/spec/acceptance/basic_swift_spec.rb +++ b/swift/spec/acceptance/basic_swift_spec.rb @@ -82,4 +82,91 @@ class { end end + + context 'Using swiftinit service provider' do + + it 'should work with no errors' do + swiftinit_pp= <<-EOS + include ::openstack_integration + include ::openstack_integration::repos + include ::openstack_integration::rabbitmq + include ::openstack_integration::mysql + include ::openstack_integration::keystone + + package { 'curl': ensure => present } + + class { '::memcached': + listen_ip => '127.0.0.1', + } + + # Swift resources + class { '::swift': + # not sure how I want to deal with this shared secret + swift_hash_suffix => 'secrete', + package_ensure => latest, + } + class { '::swift::keystone::auth': + password => 'a_big_secret', + } + # === Configure Storage + class { '::swift::storage': + storage_local_net_ip => '127.0.0.1', + } + # create xfs partitions on a loopback device and mounts them + swift::storage::loopback { '2': + require => Class['swift'], + } + # sets up storage nodes which is composed of a single + # device that contains an endpoint for an object, account, and container + swift::storage::node { '2': + mnt_base_dir => '/srv/node', + weight => 1, + manage_ring => true, + zone => '2', + storage_local_net_ip => '127.0.0.1', + require => Swift::Storage::Loopback[2] , + } + class { '::swift::storage::account': + service_provider => 'swiftinit', + } + class { '::swift::storage::container': + service_provider => 'swiftinit', + } + class { '::swift::storage::object': + service_provider => 'swiftinit', + } + class { '::swift::ringbuilder': + part_power => '18', + replicas => '1', + min_part_hours => 1, + require => Class['swift'], + } + class { '::swift::proxy': + proxy_local_net_ip => '127.0.0.1', + pipeline => ['healthcheck', 'cache', 'tempauth', 'proxy-server'], + account_autocreate => true, + require => Class['swift::ringbuilder'], + service_provider => 'swiftinit', + } + class { '::swift::proxy::authtoken': + admin_password => 'a_big_secret', + } + class {'::swift::objectexpirer': + interval => 600, + service_provider => 'swiftinit', + } + class { ['::swift::proxy::healthcheck', '::swift::proxy::cache', '::swift::proxy::tempauth']: } + EOS + + # Run one time to catch any errors upgrading to swiftinit service provider + apply_manifest(swiftinit_pp, :catch_failures => true) + # The second run tests idempotency + apply_manifest(swiftinit_pp, :catch_changes => true) + + end + + describe port(8080) do + it { is_expected.to be_listening.with('tcp') } + end + end end diff --git a/swift/spec/classes/swift_keystone_auth_spec.rb b/swift/spec/classes/swift_keystone_auth_spec.rb index 7fc3c0e65..b4479c038 100644 --- a/swift/spec/classes/swift_keystone_auth_spec.rb +++ b/swift/spec/classes/swift_keystone_auth_spec.rb @@ -57,14 +57,14 @@ it { is_expected.to contain_keystone_role(role_name).with_ensure('present') } end - it { is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}").with( + it { is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}::object-store").with( :ensure => 'present', :public_url => params[:public_url], :admin_url => params[:admin_url], :internal_url => params[:internal_url], )} - it { is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}_s3").with( + it { is_expected.to contain_keystone_endpoint("#{params[:region]}/#{params[:auth_name]}_s3::s3").with( :ensure => 'present', :public_url => params[:public_url_s3], :admin_url => params[:admin_url_s3], @@ -76,7 +76,7 @@ params.merge!(:configure_endpoint => false) end - it { is_expected.to_not contain_keystone_endpoint('RegionOne/swift') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/swift::object-store') } end context 'when disabling S3 endpoint' do @@ -84,8 +84,8 @@ params.merge!(:configure_s3_endpoint => false) end - it { is_expected.to_not contain_keystone_service('swift_s3') } - it { is_expected.to_not contain_keystone_endpoint('RegionOne/swift_s3') } + it { is_expected.to_not contain_keystone_service('swift_s3::s3') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/swift_s3::s3') } end end @@ -114,14 +114,14 @@ default_params.merge( params ) end - it { is_expected.to contain_keystone_endpoint("#{p[:region]}/#{p[:auth_name]}").with( + it { is_expected.to contain_keystone_endpoint("#{p[:region]}/#{p[:auth_name]}::object-store").with( :ensure => 'present', :public_url => "#{p[:public_protocol]}://#{p[:public_address]}:#{p[:public_port]}/v1/#{p[:endpoint_prefix]}_%(tenant_id)s", :admin_url => "#{p[:admin_protocol]}://#{p[:admin_address]}:#{p[:port]}", :internal_url => "#{p[:internal_protocol]}://#{p[:internal_address]}:#{p[:port]}/v1/#{p[:endpoint_prefix]}_%(tenant_id)s" )} - it { is_expected.to contain_keystone_endpoint("#{p[:region]}/#{p[:auth_name]}_s3").with( + it { is_expected.to contain_keystone_endpoint("#{p[:region]}/#{p[:auth_name]}_s3::s3").with( :ensure => 'present', :public_url => "#{p[:public_protocol]}://#{p[:public_address]}:#{p[:port]}", :admin_url => "#{p[:admin_protocol]}://#{p[:admin_address]}:#{p[:port]}", @@ -146,13 +146,13 @@ :roles => ['admin'], )} - it { is_expected.to contain_keystone_service(p[:auth_name]).with( + it { is_expected.to contain_keystone_service("#{p[:auth_name]}::object-store").with( :ensure => 'present', :type => 'object-store', :description => 'Openstack Object-Store Service' )} - it { is_expected.to contain_keystone_service("#{p[:auth_name]}_s3").with( + it { is_expected.to contain_keystone_service("#{p[:auth_name]}_s3::s3").with( :ensure => 'present', :type => 's3', :description => 'Openstack S3 Service' @@ -192,12 +192,12 @@ is_expected.to contain_keystone_user_role('swift@services') end it 'configures correct service name' do - is_expected.to contain_keystone_service('swift_service') - is_expected.to contain_keystone_service('swift_service_s3') + is_expected.to contain_keystone_service('swift_service::object-store') + is_expected.to contain_keystone_service('swift_service_s3::s3') end it 'configures correct endpoint name' do - is_expected.to contain_keystone_endpoint('RegionOne/swift_service') - is_expected.to contain_keystone_endpoint('RegionOne/swift_service_s3') + is_expected.to contain_keystone_endpoint('RegionOne/swift_service::object-store') + is_expected.to contain_keystone_endpoint('RegionOne/swift_service_s3::s3') end end diff --git a/swift/spec/classes/swift_objectexpirer_spec.rb b/swift/spec/classes/swift_objectexpirer_spec.rb index 43d7ea34f..5c23a8ca0 100644 --- a/swift/spec/classes/swift_objectexpirer_spec.rb +++ b/swift/spec/classes/swift_objectexpirer_spec.rb @@ -85,6 +85,18 @@ it_configures 'swift-object-expirer' end + + context 'on debian using swiftinit service provider' do + before do + params.merge!({ :service_provider => 'swiftinit' }) + end + + before do + platform_params.merge!({ :service_provider => 'swiftinit' }) + end + + it_configures 'swift-object-expirer' + end end context 'on RedHat platforms' do @@ -111,5 +123,19 @@ it_configures 'swift-object-expirer' end + + context 'on redhat using swiftinit service provider' do + before do + params.merge!({ :service_provider => 'swiftinit' }) + end + + let :platform_params do + { :object_expirer_package_name => 'openstack-swift-proxy', + :service_name => 'swift-object-expirer', + :service_provider => 'swiftinit' } + end + + it_configures 'swift-object-expirer' + end end end diff --git a/swift/spec/classes/swift_proxy_ceilometer_spec.rb b/swift/spec/classes/swift_proxy_ceilometer_spec.rb index abfa628cb..a9015de69 100644 --- a/swift/spec/classes/swift_proxy_ceilometer_spec.rb +++ b/swift/spec/classes/swift_proxy_ceilometer_spec.rb @@ -22,7 +22,8 @@ class { "swift": describe "when using default parameters" do it { is_expected.to contain_file(fragment_file).with_content(/[filter:ceilometer]/) } - it { is_expected.to contain_file(fragment_file).with_content(/use = egg:ceilometer#swift/) } + it { is_expected.to contain_file(fragment_file).with_content(/paste.filter_factory = ceilometermiddleware.swift:filter_factory/) } + it { is_expected.to contain_file(fragment_file).with_content(/url = rabbit:\/\/guest:guest@127.0.0.1:5672\//) } if Puppet.version.to_f < 4.0 it { is_expected.to contain_concat__fragment('swift_ceilometer').with_require('Class[Ceilometer]')} else @@ -34,9 +35,42 @@ class { "swift": describe "when overriding default parameters" do let :params do - { :group => 'www-data' } + { :group => 'www-data', + :rabbit_user => 'user_1', + :rabbit_password => 'user_1_passw', + :rabbit_host => '1.1.1.1', + :rabbit_port => '5673', + :rabbit_virtual_host => 'rabbit', + :driver => 'messagingv2', + :topic => 'notifications', + :control_exchange => 'swift', + } end - it { is_expected.to contain_user('swift').with_groups('www-data') } + + context 'with single rabbit host' do + it { is_expected.to contain_user('swift').with_groups('www-data') } + it { is_expected.to contain_file(fragment_file).with_content(/[filter:ceilometer]/) } + it { is_expected.to contain_file(fragment_file).with_content(/paste.filter_factory = ceilometermiddleware.swift:filter_factory/) } + it { is_expected.to contain_file(fragment_file).with_content(/url = rabbit:\/\/user_1:user_1_passw@1.1.1.1:5673\/rabbit/) } + it { is_expected.to contain_file(fragment_file).with_content(/driver = messagingv2/) } + it { is_expected.to contain_file(fragment_file).with_content(/topic = notifications/) } + it { is_expected.to contain_file(fragment_file).with_content(/control_exchange = swift/) } + end + + context 'with multiple rabbit hosts' do + before do + params.merge!({ :rabbit_hosts => ['127.0.0.1:5672', '127.0.0.2:5672'] }) + end + + it { is_expected.to contain_user('swift').with_groups('www-data') } + it { is_expected.to contain_file(fragment_file).with_content(/[filter:ceilometer]/) } + it { is_expected.to contain_file(fragment_file).with_content(/paste.filter_factory = ceilometermiddleware.swift:filter_factory/) } + it { is_expected.to contain_file(fragment_file).with_content(/url = rabbit:\/\/user_1:user_1_passw@127.0.0.1:5672,127.0.0.2:5672\/rabbit/) } + it { is_expected.to contain_file(fragment_file).with_content(/driver = messagingv2/) } + it { is_expected.to contain_file(fragment_file).with_content(/topic = notifications/) } + it { is_expected.to contain_file(fragment_file).with_content(/control_exchange = swift/) } + end + end end diff --git a/swift/spec/classes/swift_proxy_spec.rb b/swift/spec/classes/swift_proxy_spec.rb index 23f5f5a66..c285d1879 100644 --- a/swift/spec/classes/swift_proxy_spec.rb +++ b/swift/spec/classes/swift_proxy_spec.rb @@ -37,13 +37,13 @@ class { swift: swift_hash_suffix => string }" {:proxy_local_net_ip => '127.0.0.1'} end - it { is_expected.to contain_service('swift-proxy').with( - {:ensure => 'running', - :provider => 'upstart', - :enable => true, - :hasstatus => true, - :subscribe => 'Concat[/etc/swift/proxy-server.conf]', - :tag => 'swift-service', + it { is_expected.to contain_service('swift-proxy-server').with( + {:ensure => 'running', + :provider => 'upstart', + :enable => true, + :hasstatus => true, + :subscribe => 'Concat[/etc/swift/proxy-server.conf]', + :tag => 'swift-service', } )} it { is_expected.to contain_file('/etc/swift/proxy-server.conf').with( @@ -232,7 +232,7 @@ class { swift: swift_hash_suffix => string } end end - shared_examples_for 'swift-proxy' do + shared_examples_for 'swift-proxy-server' do let :params do { :proxy_local_net_ip => '127.0.0.1' } end @@ -244,14 +244,13 @@ class { swift: swift_hash_suffix => string } params.merge!(param_hash) end - it 'configures swift-proxy service' do - is_expected.to contain_service('swift-proxy').with( - :ensure => (param_hash[:manage_service] && param_hash[:enabled]) ? 'running' : 'stopped', - :name => platform_params[:service_name], - :provider => platform_params[:service_provider], - :enable => param_hash[:enabled], - :hasstatus => true, - :subscribe => 'Concat[/etc/swift/proxy-server.conf]' + it 'configures swift-proxy-server service' do + is_expected.to contain_service('swift-proxy-server').with( + :name => platform_params['swift-proxy-server'], + :ensure => (param_hash[:manage_service] && param_hash[:enabled]) ? 'running' : 'stopped', + :enable => param_hash[:enabled], + :provider => platform_params['service_provider'], + :tag => 'swift-service', ) end end @@ -264,14 +263,15 @@ class { swift: swift_hash_suffix => string } :enabled => false }) end - it 'configures swift-proxy service' do - is_expected.to contain_service('swift-proxy').with( - :ensure => nil, - :name => platform_params[:service_name], - :provider => platform_params[:service_provider], - :enable => false, - :hasstatus => true, - :subscribe => 'Concat[/etc/swift/proxy-server.conf]' + it 'configures swift-proxy-server service' do + + is_expected.to contain_service('swift-proxy-server').with( + :ensure => nil, + :name => platform_params['swift-proxy-server'], + :provider => platform_params['service_provider'], + :enable => false, + :hasstatus => true, + :subscribe => 'Concat[/etc/swift/proxy-server.conf]' ) end end @@ -285,11 +285,24 @@ class { swift: swift_hash_suffix => string } end let :platform_params do - { :service_name => 'swift-proxy', - :service_provider => 'upstart' } + { 'swift-proxy-server' => 'swift-proxy', + 'service_provider' => 'upstart' + } end + it_configures 'swift-proxy-server' + + context 'on Debian platforms using swiftinit service provider' do + before do + params.merge!({ :service_provider => 'swiftinit' }) + end - it_configures 'swift-proxy' + let :platform_params do + { 'swift-proxy-server' => 'swift-proxy-server', + 'service_provider' => 'swiftinit' + } + end + it_configures 'swift-proxy-server' + end end context 'on RedHat platforms' do @@ -300,10 +313,23 @@ class { swift: swift_hash_suffix => string } end let :platform_params do - { :service_name => 'openstack-swift-proxy', - :service_provider => nil } + { + 'swift-proxy-server' => 'openstack-swift-proxy', + } end + it_configures 'swift-proxy-server' - it_configures 'swift-proxy' + context 'on Redhat platforms using swiftinit service provider' do + before do + params.merge!({ :service_provider => 'swiftinit' }) + end + + let :platform_params do + { 'swift-proxy-server' => 'swift-proxy-server', + 'service_provider' => 'swiftinit' + } + end + it_configures 'swift-proxy-server' + end end end diff --git a/swift/spec/classes/swift_storage_account_spec.rb b/swift/spec/classes/swift_storage_account_spec.rb index c1473ce92..25647eb97 100644 --- a/swift/spec/classes/swift_storage_account_spec.rb +++ b/swift/spec/classes/swift_storage_account_spec.rb @@ -39,6 +39,7 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" :name => service_name, :ensure => (param_hash[:manage_service] && param_hash[:enabled]) ? 'running' : 'stopped', :enable => param_hash[:enabled], + :provider => platform_params[:service_provider], :tag => 'swift-service', ) end @@ -75,15 +76,35 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" let :platform_params do { :service_names => { - 'swift-account' => 'swift-account', + 'swift-account-server' => 'swift-account', 'swift-account-replicator' => 'swift-account-replicator', 'swift-account-reaper' => 'swift-account-reaper', 'swift-account-auditor' => 'swift-account-auditor' - } + }, + :service_provider => 'upstart' } end it_configures 'swift-storage-account' + context 'on Debian platforms using swiftinit service provider' do + + before do + params.merge!({ :service_provider => 'swiftinit' }) + end + + let :platform_params do + { :service_names => { + 'swift-account-server' => 'swift-account-server', + 'swift-account-replicator' => 'swift-account-replicator', + 'swift-account-reaper' => 'swift-account-reaper', + 'swift-account-auditor' => 'swift-account-auditor', + }, + :service_provider => 'swiftinit' + } + end + + it_configures 'swift-storage-account' + end end context 'on RedHat platforms' do @@ -94,14 +115,34 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" let :platform_params do { :service_names => { - 'swift-account' => 'openstack-swift-account', + 'swift-account-server' => 'openstack-swift-account', 'swift-account-replicator' => 'openstack-swift-account-replicator', 'swift-account-reaper' => 'openstack-swift-account-reaper', 'swift-account-auditor' => 'openstack-swift-account-auditor' - } + }, + } end it_configures 'swift-storage-account' + context 'on redhat using swiftinit service provider' do + + before do + params.merge!({ :service_provider => 'swiftinit' }) + end + + let :platform_params do + { :service_names => { + 'swift-account-server' => 'swift-account-server', + 'swift-account-replicator' => 'swift-account-replicator', + 'swift-account-reaper' => 'swift-account-reaper', + 'swift-account-auditor' => 'swift-account-auditor', + }, + :service_provider => 'swiftinit' + } + end + + it_configures 'swift-storage-account' + end end end diff --git a/swift/spec/classes/swift_storage_all_spec.rb b/swift/spec/classes/swift_storage_all_spec.rb index 4139ee2b5..fdd6cd9d3 100644 --- a/swift/spec/classes/swift_storage_all_spec.rb +++ b/swift/spec/classes/swift_storage_all_spec.rb @@ -60,7 +60,7 @@ ['object', 'container', 'account'].each do |type| it { is_expected.to contain_package("swift-#{type}").with_ensure('present') } - it { is_expected.to contain_service("swift-#{type}").with( + it { is_expected.to contain_service("swift-#{type}-server").with( {:provider => 'upstart', :ensure => 'running', :enable => true, @@ -147,7 +147,7 @@ end ['object', 'container', 'account'].each do |type| it { is_expected.to contain_package("swift-#{type}").with_ensure('present') } - it { is_expected.to contain_service("swift-#{type}").with( + it { is_expected.to contain_service("swift-#{type}-server").with( {:provider => nil, :ensure => 'running', :enable => true, diff --git a/swift/spec/classes/swift_storage_container_spec.rb b/swift/spec/classes/swift_storage_container_spec.rb index 429e1db6e..5434b019b 100644 --- a/swift/spec/classes/swift_storage_container_spec.rb +++ b/swift/spec/classes/swift_storage_container_spec.rb @@ -36,10 +36,11 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" it 'configures services' do platform_params[:service_names].each do |service_alias, service_name| is_expected.to contain_service(service_alias).with( - :name => service_name, - :ensure => (param_hash[:manage_service] && param_hash[:enabled]) ? 'running' : 'stopped', - :enable => param_hash[:enabled], - :tag => 'swift-service', + :name => service_name, + :ensure => (param_hash[:manage_service] && param_hash[:enabled]) ? 'running' : 'stopped', + :enable => param_hash[:enabled], + :provider => platform_params[:service_provider], + :tag => 'swift-service', ) end end @@ -75,29 +76,36 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" let :platform_params do { :service_names => { - 'swift-container' => 'swift-container', + 'swift-container-server' => 'swift-container', 'swift-container-replicator' => 'swift-container-replicator', 'swift-container-updater' => 'swift-container-updater', 'swift-container-auditor' => 'swift-container-auditor' - } + }, + :service_provider => 'upstart' } end it_configures 'swift-storage-container' - context 'Ubuntu specific resources' do - it 'configures sync' do - is_expected.to contain_service('swift-container-sync').with( - :ensure => 'running', - :enable => true, - :provider => 'upstart', - :require => 'File[/etc/init/swift-container-sync.conf]', - ) - is_expected.to contain_file('/etc/init/swift-container-sync.conf').with( - :source => 'puppet:///modules/swift/swift-container-sync.conf.upstart', - :require => 'Package[swift-container]' - ) + context 'on debian using swiftinit service provider' do + + before do + params.merge!({ :service_provider => 'swiftinit' }) + end + + let :platform_params do + { :service_names => { + 'swift-container-server' => 'swift-container-server', + 'swift-container-replicator' => 'swift-container-replicator', + 'swift-container-updater' => 'swift-container-updater', + 'swift-container-auditor' => 'swift-container-auditor', + 'swift-container-sync' => 'swift-container-sync' + }, + :service_provider => 'swiftinit' + } end + + it_configures 'swift-storage-container' end end @@ -109,7 +117,7 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" let :platform_params do { :service_names => { - 'swift-container' => 'openstack-swift-container', + 'swift-container-server' => 'openstack-swift-container', 'swift-container-replicator' => 'openstack-swift-container-replicator', 'swift-container-updater' => 'openstack-swift-container-updater', 'swift-container-auditor' => 'openstack-swift-container-auditor' @@ -119,23 +127,24 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" it_configures 'swift-storage-container' - context 'RedHat specific resources' do - before do - params.merge!({ :allowed_sync_hosts => ['127.0.0.1', '10.1.0.1', '10.1.0.2'] }) - end + context 'on redhat using swiftinit service provider' do - let :pre_condition do - "class { 'swift': swift_hash_suffix => 'foo' } - class { 'swift::storage::all': storage_local_net_ip => '10.0.0.1' }" + before do + params.merge!({ :service_provider => 'swiftinit' }) end - let :fragment_file do - "/var/lib/puppet/concat/_etc_swift_container-server.conf/fragments/00_swift-container-6001" + let :platform_params do + { :service_names => { + 'swift-container-server' => 'swift-container-server', + 'swift-container-replicator' => 'swift-container-replicator', + 'swift-container-updater' => 'swift-container-updater', + 'swift-container-auditor' => 'swift-container-auditor', + }, + :service_provider => 'swiftinit' + } end - it { - is_expected.to contain_file(fragment_file).with_content(/^allowed_sync_hosts = 127.0.0.1,10.1.0.1,10.1.0.2$/) - } + it_configures 'swift-storage-container' end end end diff --git a/swift/spec/classes/swift_storage_object_spec.rb b/swift/spec/classes/swift_storage_object_spec.rb index 7cbd1c2ae..c770b415a 100644 --- a/swift/spec/classes/swift_storage_object_spec.rb +++ b/swift/spec/classes/swift_storage_object_spec.rb @@ -37,10 +37,11 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" it 'configures services' do platform_params[:service_names].each do |service_alias, service_name| is_expected.to contain_service(service_alias).with( - :name => service_name, - :ensure => (param_hash[:manage_service] && param_hash[:enabled]) ? 'running' : 'stopped', - :enable => param_hash[:enabled], - :tag => 'swift-service', + :name => service_name, + :ensure => (param_hash[:manage_service] && param_hash[:enabled]) ? 'running' : 'stopped', + :enable => param_hash[:enabled], + :provider => platform_params[:service_provider], + :tag => 'swift-service', ) end end @@ -76,15 +77,37 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" let :platform_params do { :service_names => { - 'swift-object' => 'swift-object', + 'swift-object-server' => 'swift-object', 'swift-object-replicator' => 'swift-object-replicator', 'swift-object-updater' => 'swift-object-updater', 'swift-object-auditor' => 'swift-object-auditor' - } + }, + :service_provider => 'upstart' } end it_configures 'swift-storage-object' + context 'on debian using swiftinit service provider' do + + before do + params.merge!({ :service_provider => 'swiftinit' }) + end + + let :platform_params do + { :service_names => { + 'swift-object-server' => 'swift-object-server', + 'swift-object-replicator' => 'swift-object-replicator', + 'swift-object-updater' => 'swift-object-updater', + 'swift-object-auditor' => 'swift-object-auditor', + }, + :service_provider => 'swiftinit' + } + end + + it_configures 'swift-storage-object' + end + + end context 'on RedHat platforms' do @@ -95,7 +118,7 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" let :platform_params do { :service_names => { - 'swift-object' => 'openstack-swift-object', + 'swift-object-server' => 'openstack-swift-object', 'swift-object-replicator' => 'openstack-swift-object-replicator', 'swift-object-updater' => 'openstack-swift-object-updater', 'swift-object-auditor' => 'openstack-swift-object-auditor' @@ -104,5 +127,24 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" end it_configures 'swift-storage-object' + context 'on redhat using swiftinit service provider' do + + before do + params.merge!({ :service_provider => 'swiftinit' }) + end + + let :platform_params do + { :service_names => { + 'swift-object-server' => 'swift-object-server', + 'swift-object-replicator' => 'swift-object-replicator', + 'swift-object-updater' => 'swift-object-updater', + 'swift-object-auditor' => 'swift-object-auditor', + }, + :service_provider => 'swiftinit' + } + end + + it_configures 'swift-storage-object' + end end end diff --git a/swift/spec/defines/swift_storage_generic_spec.rb b/swift/spec/defines/swift_storage_generic_spec.rb index 3b1ef1678..41c4b5b50 100644 --- a/swift/spec/defines/swift_storage_generic_spec.rb +++ b/swift/spec/defines/swift_storage_generic_spec.rb @@ -18,9 +18,10 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" end - let :default_params do - {:package_ensure => 'present', - :service_provider => 'upstart'} + let :params do + { :package_ensure => 'present', + :enabled => true, + :manage_service => true } end describe 'with an invalid title' do @@ -30,45 +31,158 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" it_raises 'a Puppet::Error', /does not match/ end - ['account', 'object', 'container'].each do |t| - [{}, - {:package_ensure => 'latest', - :service_provider => 'init'} - ].each do |param_set| - describe "when #{param_set == {} ? "using default" : "specifying"} class parameters" do - let :title do - t - end - let :param_hash do - default_params.merge(param_set) - end - let :params do - param_set + shared_examples_for 'swift-storage-generic' do + %w(account object container).each do |t| + [{}, + { :package_ensure => 'latest' } + ].each do |param_set| + describe "when #{param_set == {} ? 'using default' : 'specifying'} class parameters" do + before do + params.merge!(param_set) + end + + let :title do + t + end + + [{ :enabled => true, :manage_service => true }, + { :enabled => false, :manage_service => true }].each do |param_hash_manage| + context "when service is_expected.to be #{param_hash_manage[:enabled] ? 'enabled' : 'disabled'}" do + before do + params.merge!(param_hash_manage) + end + + it do + is_expected.to contain_package("swift-#{t}").with( + :ensure => params[:package_ensure], + :tag => ['openstack', 'swift-package'] + ) + end + it do + is_expected.to contain_service("swift-#{t}-server").with( + :name => platform_params["swift-#{t}-server"], + :ensure => (param_hash_manage[:manage_service] && param_hash_manage[:enabled]) ? 'running' : 'stopped', + :enable => param_hash_manage[:enabled], + :provider => platform_params['service_provider'], + :tag => 'swift-service' + ) + end + it do + is_expected.to contain_service("swift-#{t}-replicator").with( + :name => platform_params["swift-#{t}-replicator"], + :ensure => (param_hash_manage[:manage_service] && param_hash_manage[:enabled]) ? 'running' : 'stopped', + :enable => param_hash_manage[:enabled], + :provider => platform_params['service_provider'], + :tag => 'swift-service' + ) + end + it do + is_expected.to contain_service("swift-#{t}-auditor").with( + :name => platform_params["swift-#{t}-auditor"], + :ensure => (param_hash_manage[:manage_service] && param_hash_manage[:enabled]) ? 'running' : 'stopped', + :enable => param_hash_manage[:enabled], + :provider => platform_params['service_provider'], + :tag => 'swift-service' + ) + end + it do + is_expected.to contain_file("/etc/swift/#{t}-server/").with( + :ensure => 'directory', + ) + end + end + end end - it { is_expected.to contain_package("swift-#{t}").with( - :ensure => param_hash[:package_ensure], - :tag => ['openstack', 'swift-package'], - )} - it { is_expected.to contain_service("swift-#{t}").with( - :ensure => 'running', - :enable => true, - :hasstatus => true, - :provider => param_hash[:service_provider], - :tag => 'swift-service', - )} - it { is_expected.to contain_service("swift-#{t}-replicator").with( - :ensure => 'running', - :enable => true, - :hasstatus => true, - :provider => param_hash[:service_provider], - :tag => 'swift-service', - )} - it { is_expected.to contain_file("/etc/swift/#{t}-server/").with( - :ensure => 'directory', - )} end - # TODO - I do not want to add tests for the upstart stuff - # I need to check the tickets and see if this stuff is fixed + end + end + + context 'on Debian platforms' do + let :facts do + { :operatingsystem => 'Ubuntu', + :osfamily => 'Debian' } + end + + let :platform_params do + { 'swift-account-server' => 'swift-account', + 'swift-account-replicator' => 'swift-account-replicator', + 'swift-account-auditor' => 'swift-account-auditor', + 'swift-container-server' => 'swift-container', + 'swift-container-replicator' => 'swift-container-replicator', + 'swift-container-auditor' => 'swift-container-auditor', + 'swift-object-server' => 'swift-object', + 'swift-object-replicator' => 'swift-object-replicator', + 'swift-object-auditor' => 'swift-object-auditor', + 'service_provider' => 'upstart' + } + end + + it_configures 'swift-storage-generic' + + context 'on Debian platforms using swiftinit service provider' do + before do + params.merge!(:service_provider => 'swiftinit') + end + + let :platform_params do + { 'swift-account-server' => 'swift-account-server', + 'swift-account-replicator' => 'swift-account-replicator', + 'swift-account-auditor' => 'swift-account-auditor', + 'swift-container-server' => 'swift-container-server', + 'swift-container-replicator' => 'swift-container-replicator', + 'swift-container-auditor' => 'swift-container-auditor', + 'swift-object-server' => 'swift-object-server', + 'swift-object-replicator' => 'swift-object-replicator', + 'swift-object-auditor' => 'swift-object-auditor', + 'service_provider' => 'swiftinit', + } + end + + it_configures 'swift-storage-generic' + end + end + + context 'on Redhat platforms' do + let :facts do + { :operatingsystem => 'Redhat', + :osfamily => 'Redhat' } + end + + let :platform_params do + { 'swift-account-server' => 'openstack-swift-account', + 'swift-account-replicator' => 'openstack-swift-account-replicator', + 'swift-account-auditor' => 'openstack-swift-account-auditor', + 'swift-container-server' => 'openstack-swift-container', + 'swift-container-replicator' => 'openstack-swift-container-replicator', + 'swift-container-auditor' => 'openstack-swift-container-auditor', + 'swift-object-server' => 'openstack-swift-object', + 'swift-object-replicator' => 'openstack-swift-object-replicator', + 'swift-object-auditor' => 'openstack-swift-object-auditor', + } + end + + it_configures 'swift-storage-generic' + + context 'on Redhat platforms using swiftinit service provider' do + before do + params.merge!(:service_provider => 'swiftinit') + end + + let :platform_params do + { 'swift-account-server' => 'swift-account-server', + 'swift-account-replicator' => 'swift-account-replicator', + 'swift-account-auditor' => 'swift-account-auditor', + 'swift-container-server' => 'swift-container-server', + 'swift-container-replicator' => 'swift-container-replicator', + 'swift-container-auditor' => 'swift-container-auditor', + 'swift-object-server' => 'swift-object-server', + 'swift-object-replicator' => 'swift-object-replicator', + 'swift-object-auditor' => 'swift-object-auditor', + 'service_provider' => 'swiftinit', + } + end + + it_configures 'swift-storage-generic' end end end diff --git a/swift/spec/defines/swift_storage_server_spec.rb b/swift/spec/defines/swift_storage_server_spec.rb index c89a9b580..d3be56cf4 100644 --- a/swift/spec/defines/swift_storage_server_spec.rb +++ b/swift/spec/defines/swift_storage_server_spec.rb @@ -53,13 +53,13 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' }" end it { is_expected.to contain_package("swift-#{t}").with_ensure('present') } - it { is_expected.to contain_service("swift-#{t}").with( - :ensure => 'running', - :enable => true, - :hasstatus => true + it { is_expected.to contain_service("swift-#{t}-server").with( + :ensure => 'running', + :enable => true, + :hasstatus => true, )} let :fragment_file do - "/var/lib/puppet/concat/_etc_swift_#{t}-server_#{title}.conf/fragments/00_swift-#{t}-#{title}" + "/var/lib/puppet/concat/_etc_swift_#{t}-server.conf/fragments/00_swift-#{t}-#{title}" end describe 'when parameters are overridden' do @@ -141,7 +141,7 @@ class { 'swift::storage': storage_local_net_ip => '10.0.0.1' } swift_#{t}_config { 'foo/bar': value => 'foo' } " end - it { is_expected.to contain_concat("/etc/swift/#{t}-server/#{title}.conf").that_comes_before("Swift_#{t}_config[foo/bar]") } + it { is_expected.to contain_concat("/etc/swift/#{t}-server.conf").that_comes_before("Swift_#{t}_config[foo/bar]") } end describe "when log_requests is turned off" do let :params do req_params.merge({:log_requests => false}) end diff --git a/swift/spec/unit/puppet/provider/service/swiftinit_spec.rb b/swift/spec/unit/puppet/provider/service/swiftinit_spec.rb new file mode 100644 index 000000000..535652460 --- /dev/null +++ b/swift/spec/unit/puppet/provider/service/swiftinit_spec.rb @@ -0,0 +1,71 @@ +#! /usr/bin/env ruby +## +## Unit testing for the swiftinit service provider +## + +require 'spec_helper' + +provider_class = Puppet::Type.type(:service).provider(:swiftinit) + +describe provider_class do + + + before(:each) do + # Create a mock resource + @resource = stub 'resource' + + @provider = provider_class.new + + # A catch all; no parameters set + @resource.stubs(:[]).returns(nil) + + # But set name, source and path + @resource.stubs(:[]).with(:name).returns "swift-object-server" + @resource.stubs(:[]).with(:ensure).returns :enable + @resource.stubs(:[]).with(:pattern).returns "swift-object" + @resource.stubs(:[]).with(:manifest).returns "object-server" + @resource.stubs(:ref).returns "Service[myservice]" + + @provider.resource = @resource + + @provider.stubs(:command).with(:systemctl_run).returns "systemctl_run" + + @provider.stubs(:systemctl_run) + + end + + it "should have an status method" do + expect(@provider).to respond_to(:status) + end + + it "should have an start method" do + expect(@provider).to respond_to(:start) + end + + it "should have an stop method" do + expect(@provider).to respond_to(:stop) + end + + it "should have an restart method" do + expect(@provider).to respond_to(:restart) + end + + it "should have an refresh method" do + expect(@provider).to respond_to(:refresh) + end + + it "should have an enabled? method" do + expect(@provider).to respond_to(:enabled?) + end + + it "should have an enable method" do + expect(@provider).to respond_to(:enable) + end + + it "should have a disable method" do + expect(@provider).to respond_to(:disable) + end + +end + +##### TODO figure out how to stub out files and test each method more. diff --git a/swift/spec/unit/puppet/provider/swift_ring_builder_spec.rb b/swift/spec/unit/puppet/provider/swift_ring_builder_spec.rb index 077201841..cad198e9c 100644 --- a/swift/spec/unit/puppet/provider/swift_ring_builder_spec.rb +++ b/swift/spec/unit/puppet/provider/swift_ring_builder_spec.rb @@ -11,6 +11,45 @@ '/etc/swift/account.builder' end + it 'should be able to lookup the local ring and build an object 2.2.2+' do + File.expects(:exists?).with(builder_file_path).returns(true) + provider_class.expects(:builder_file_path).twice.returns(builder_file_path) + # Swift 1.8 output + provider_class.expects(:swift_ring_builder).returns( +'/etc/swift/account.builder, build version 3 +262144 partitions, 3 replicas, 3 zones, 3 devices, 0.00 balance +The minimum number of hours before a partition can be reassigned is 1 +The overload factor is 0.00% (0.000000) +Devices: id region zone ip address port replication ip replication port name weight partitions balance meta + 1 1 1 192.168.101.13 6002 192.168.101.13 6002 1 1.00 262144 0.00 + 2 1 2 192.168.101.14 6002 192.168.101.14 6002 1 1.00 262144 200.00 m2 + 0 1 3 192.168.101.15 6002 192.168.101.15 6002 1 1.00 262144-100.00 m2 + 3 1 1 192.168.101.16 6002 192.168.101.16 6002 1 1.00 262144-100.00 +' + ) + resources = provider_class.lookup_ring + expect(resources['192.168.101.13:6002/1']).to_not be_nil + expect(resources['192.168.101.14:6002/1']).to_not be_nil + expect(resources['192.168.101.15:6002/1']).to_not be_nil + expect(resources['192.168.101.16:6002/1']).to_not be_nil + + expect(resources['192.168.101.13:6002/1'][:id]).to eql '1' + expect(resources['192.168.101.13:6002/1'][:region]).to eql '1' + expect(resources['192.168.101.13:6002/1'][:zone]).to eql '1' + expect(resources['192.168.101.13:6002/1'][:weight]).to eql '1.00' + expect(resources['192.168.101.13:6002/1'][:partitions]).to eql '262144' + expect(resources['192.168.101.13:6002/1'][:balance]).to eql '0.00' + expect(resources['192.168.101.13:6002/1'][:meta]).to eql '' + + expect(resources['192.168.101.14:6002/1'][:id]).to eql '2' + expect(resources['192.168.101.14:6002/1'][:region]).to eql '1' + expect(resources['192.168.101.14:6002/1'][:zone]).to eql '2' + expect(resources['192.168.101.14:6002/1'][:weight]).to eql '1.00' + expect(resources['192.168.101.14:6002/1'][:partitions]).to eql '262144' + expect(resources['192.168.101.14:6002/1'][:balance]).to eql '200.00' + expect(resources['192.168.101.14:6002/1'][:meta]).to eql 'm2' + end + it 'should be able to lookup the local ring and build an object 1.8+' do File.expects(:exists?).with(builder_file_path).returns(true) provider_class.expects(:builder_file_path).twice.returns(builder_file_path) diff --git a/swift/templates/proxy/ceilometer.conf.erb b/swift/templates/proxy/ceilometer.conf.erb index 73a7d03d8..28fca3e5e 100644 --- a/swift/templates/proxy/ceilometer.conf.erb +++ b/swift/templates/proxy/ceilometer.conf.erb @@ -1,3 +1,18 @@ [filter:ceilometer] -use = egg:ceilometer#swift +paste.filter_factory = ceilometermiddleware.swift:filter_factory +<% if @rabbit_hosts.nil? %> +url = rabbit://<%= @rabbit_user %>:<%= @rabbit_password %>@<%= @rabbit_host %>:<%= @rabbit_port %>/<%= @rabbit_virtual_host %> +<% else %> +<% hosts=Array(@rabbit_hosts).join(",") -%> +url = rabbit://<%= @rabbit_user %>:<%= @rabbit_password %>@<%= hosts %>/<%= @rabbit_virtual_host %> +<% end %> +<% if @driver %> +driver = <%= @driver %> +<% end %> +<% if @topic %> +topic = <%= @topic %> +<% end %> +<% if @control_exchange %> +control_exchange = <%= @control_exchange %> +<% end %> diff --git a/tripleo/.fixtures.yml b/tripleo/.fixtures.yml index e3ab8f9cd..69512da84 100644 --- a/tripleo/.fixtures.yml +++ b/tripleo/.fixtures.yml @@ -2,5 +2,26 @@ fixtures: repositories: 'firewall': 'git://github.com/puppetlabs/puppetlabs-firewall.git' 'stdlib': 'git://github.com/puppetlabs/puppetlabs-stdlib.git' + 'midonet': + repo: 'git://github.com/midonet/puppet-midonet.git' + ref: 'v2015.06.7' + 'tomcat': + repo: 'git://github.com/puppetlabs/puppetlabs-tomcat.git' + ref: '1.3.2' + 'inifile': + repo: 'git://github.com/puppetlabs/puppetlabs-inifile.git' + ref: '1.4.2' + 'cassandra': + repo: 'git://github.com/locp/cassandra.git' + ref: '1.9.2' + 'zookeeper': + repo: 'git://github.com/deric/puppet-zookeeper.git' + ref: 'v0.3.9' + 'datacat': + repo: 'git://github.com/richardc/puppet-datacat' + ref: '0.6.2' + 'java': + repo: 'git://github.com/puppetlabs/puppetlabs-java' + ref: '1.4.2' symlinks: "tripleo": "#{source_dir}" diff --git a/tripleo/Gemfile b/tripleo/Gemfile index 6d4ce9a07..91c5f0a3e 100644 --- a/tripleo/Gemfile +++ b/tripleo/Gemfile @@ -14,6 +14,10 @@ group :development, :test do gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' gem 'webmock', :require => 'false' + # adding 'psych' explicitly + # https://github.com/bundler/bundler/issues/2068 + # TODO: drop it in a future release of 'bundle'. + gem 'psych', :require => 'false' end group :system_tests do diff --git a/tripleo/lib/facter/netmask_ipv6.rb b/tripleo/lib/facter/netmask_ipv6.rb new file mode 100644 index 000000000..526148530 --- /dev/null +++ b/tripleo/lib/facter/netmask_ipv6.rb @@ -0,0 +1,47 @@ +require 'ipaddr' + +def netmask6(value) + if value + ip = IPAddr.new('::0').mask(value) + ip.inspect.split('/')[1].gsub('>', '') + end +end + +if Facter.value('facterversion')[0].to_i < 3 + Facter::Util::IP.get_interfaces.each do |interface| + Facter.add('netmask6_' + Facter::Util::IP.alphafy(interface)) do + setcode do + tmp = [] + regex = %r{inet6\s+.*\s+(?:prefixlen)\s+(\d+)}x + output_int = Facter::Util::IP.get_output_for_interface_and_label(interface, 'netmask6') + + output_int.each_line do |line| + prefixlen = nil + matches = line.match(regex) + prefixlen = matches[1] if matches + + if prefixlen + value = netmask6(prefixlen) + tmp.push(value) + end + end + + tmp.shift if tmp + end + end + end + + Facter.add('netmask6') do + setcode do + prefixlen = nil + regex = %r{#{Facter.value(:ipaddress6)}.*?(?:prefixlen)\s*(\d+)}x + + String(Facter::Util::IP.exec_ifconfig(['2>/dev/null'])).split(/\n/).collect do |line| + matches = line.match(regex) + prefixlen = matches[1] if matches + end + + netmask6(prefixlen) if prefixlen + end + end +end diff --git a/tripleo/lib/puppet/parser/functions/extract_id.rb b/tripleo/lib/puppet/parser/functions/extract_id.rb new file mode 100644 index 000000000..61734abfa --- /dev/null +++ b/tripleo/lib/puppet/parser/functions/extract_id.rb @@ -0,0 +1,14 @@ +# Custom function to extract the index from a list. +# The list are a list of hostname, and the index is the n'th +# position of the host in list +module Puppet::Parser::Functions + newfunction(:extract_id, :type => :rvalue) do |argv| + hosts = argv[0] + if hosts.class != Array + hosts = [hosts] + end + hostname = argv[1] + hash = Hash[hosts.map.with_index.to_a] + return hash[hostname].to_i + 1 + end +end diff --git a/tripleo/lib/puppet/parser/functions/interface_for_ip.rb b/tripleo/lib/puppet/parser/functions/interface_for_ip.rb index 1c6712035..fd68be0c2 100644 --- a/tripleo/lib/puppet/parser/functions/interface_for_ip.rb +++ b/tripleo/lib/puppet/parser/functions/interface_for_ip.rb @@ -8,25 +8,31 @@ module Puppet::Parser::Functions newfunction(:interface_for_ip, :type => :rvalue, :doc => "Find the bind IP address for the provided subnet.") do |arg| if arg[0].class == String begin - ip_to_find = arg[0] + ip1 = IPAddr.new(arg[0]) Dir.foreach('/sys/class/net/') do |interface| - next if interface == '.' or interface == '..' + next if interface == '.' || interface == '..' iface_no_dash = interface.gsub('-', '_') - interface_ip = lookupvar("ipaddress_#{iface_no_dash}") - netmask = lookupvar("netmask_#{iface_no_dash}") - if not interface_ip.nil? then - ip1=IPAddr.new(interface_ip) - ip2=IPAddr.new(ip_to_find) - if ip1.mask(netmask) == ip2.mask(netmask) then - return interface - end + + if ip1.ipv4? + ipaddress_name = "ipaddress_#{iface_no_dash}" + netmask_name = "netmask_#{iface_no_dash}" + else + ipaddress_name = "ipaddress6_#{iface_no_dash}" + netmask_name = "netmask6_#{iface_no_dash}" + end + + interface_ip = lookupvar(ipaddress_name) + netmask = lookupvar(netmask_name) + unless interface_ip.nil? then + ip2 = IPAddr.new(interface_ip) + return interface if ip1.mask(netmask) == ip2.mask(netmask) end end - rescue JSON::ParserError - raise Puppet::ParseError, "Syntax error: #{arg[0]} is invalid" + rescue IPAddr::InvalidAddressError => e + raise Puppet::ParseError, "#{e}: #{arg[0]}" end else - raise Puppet::ParseError, "Syntax error: #{arg[0]} is not a String" + raise Puppet::ParseError, "Syntax error: #{arg[0]} must be a String" end return '' end diff --git a/tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb b/tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb new file mode 100644 index 000000000..814326e84 --- /dev/null +++ b/tripleo/lib/puppet/parser/functions/list_to_zookeeper_hash.rb @@ -0,0 +1,24 @@ +# Custom function to convert a list of ips to a map +# like {'ip' => xxx.xxx.xxx.xxx }. This function is needed +# because a not-so-good design of the puppet-midonet module +# and we hope to deprecate it soon. + +module Puppet::Parser::Functions + newfunction(:list_to_zookeeper_hash, :type => :rvalue, :doc => <<-EOS + This function returns Zookeper configuration list of hash + EOS + ) do |argv| + zk_list = argv[0] + if zk_list.class != Array + zk_list = [zk_list] + end + result = Array.new + zk_list.each do |zk_ip| + zk_map = Hash.new + zk_map['ip'] = zk_ip + zk_map['port'] = 2181 + result.push(zk_map) + end + return result + end +end diff --git a/tripleo/manifests/cluster/cassandra.pp b/tripleo/manifests/cluster/cassandra.pp new file mode 100644 index 000000000..deca05fcb --- /dev/null +++ b/tripleo/manifests/cluster/cassandra.pp @@ -0,0 +1,68 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# == Class: tripleo::cluster::cassandra +# +# Deploys a cassandra service that belongs to a cluster. Uses puppet-cassandra +# +# == Parameters: +# +# [*cassandra_servers*] +# (required) All the IP addresses of the cassandra cluster. +# Array of strings value. +# +# [*cassandra_ip*] +# (required) IP address of the current host. +# String value +# +# [*storage_port*] +# (optional) Inter-node cluster communication port. +# Defaults to 7000. +# +# [*ssl_storage_port*] +# (optional) SSL Inter-node cluster communication port. +# Defaults to 7001. +# +# [*client_port*] +# (optional) Cassandra client port. +# Defaults to 9042. +# +# [*client_port_thrift*] +# (optional) Cassandra client port thrift. +# Defaults to 9160. +# +class tripleo::cluster::cassandra( + $cassandra_servers, + $cassandra_ip, + $storage_port = '7000', + $ssl_storage_port = '7001', + $client_port = '9042', + $client_port_thrift = '9160' +) +{ + validate_array($cassandra_servers) + validate_ipv4_address($cassandra_ip) + + class {'::cassandra': + cluster_name => 'TripleO', + seeds => $cassandra_servers, + listen_address => $cassandra_ip, + storage_port => $storage_port, + ssl_storage_port => $ssl_storage_port, + native_transport_port => $client_port, + rpc_port => $client_port_thrift + } + +} diff --git a/tripleo/manifests/cluster/zookeeper.pp b/tripleo/manifests/cluster/zookeeper.pp new file mode 100644 index 000000000..6f4adbcd8 --- /dev/null +++ b/tripleo/manifests/cluster/zookeeper.pp @@ -0,0 +1,65 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# == Class: tripleo::cluster::zookeeper +# +# Deploys a zookeeper service that belongs to a cluster. Uses deric-zookeeper +# +# == Parameters: +# +# [*zookeeper_server_ips*] +# (required) List of IP addresses of the zookeeper cluster. +# Arrays of strings value. +# +# [*zookeeper_client_ip*] +# (required) IP address of the host where zookeeper will listen IP addresses. +# String (IPv4) value. +# +# [*zookeeper_hostnames*] +# (required) List of hostnames of the zookeeper cluster. The hostname of the +# node will be used to define the ID of the zookeeper configuration +# Array of strings value. +# + +class tripleo::cluster::zookeeper( + $zookeeper_server_ips, + $zookeeper_client_ip, + $zookeeper_hostnames +) +{ + validate_array($zookeeper_server_ips) + validate_ipv4_address($zookeeper_client_ip) + validate_array($zookeeper_hostnames) + + # TODO(devvesa) Zookeeper package should provide these paths, + # remove this lines as soon as it will. + file {['/usr/lib', '/usr/lib/zookeeper', '/usr/lib/zookeeper/bin/']: + ensure => directory + } + + file {'/usr/lib/zookeeper/bin/zkEnv.sh': + ensure => link, + target => '/usr/libexec/zkEnv.sh' + } + + class {'::zookeeper': + servers => $zookeeper_server_ips, + client_ip => $zookeeper_client_ip, + id => extract_id($zookeeper_hostnames, $::hostname), + cfg_dir => '/etc/zookeeper/conf', + } + + File['/usr/lib/zookeeper/bin/zkEnv.sh'] -> Class['::zookeeper'] +} diff --git a/tripleo/manifests/loadbalancer.pp b/tripleo/manifests/loadbalancer.pp index 16a4f80a9..efca05700 100644 --- a/tripleo/manifests/loadbalancer.pp +++ b/tripleo/manifests/loadbalancer.pp @@ -238,6 +238,10 @@ # (optional) Enable or not Redis binding # Defaults to false # +# [*midonet_api*] +# (optional) Enable or not MidoNet API binding +# Defaults to false +# class tripleo::loadbalancer ( $controller_virtual_ip, $control_virtual_interface, @@ -288,6 +292,7 @@ $mysql_clustercheck = false, $rabbitmq = false, $redis = false, + $midonet_api = false, ) { if !$controller_host and !$controller_hosts { @@ -814,6 +819,7 @@ bind => $nova_novnc_bind_opts, options => { 'balance' => 'source', + 'timeout' => [ 'tunnel 1h' ], }, collect_exported => false, } @@ -1001,4 +1007,23 @@ } } + $midonet_api_vip = hiera('midonet_api_vip', $controller_virtual_ip) + $midonet_bind_opts = { + "${midonet_api_vip}:8081" => [], + "${public_virtual_ip}:8081" => [], + } + + if $midonet_api { + haproxy::listen { 'midonet_api': + bind => $midonet_bind_opts, + collect_exported => false, + } + haproxy::balancermember { 'midonet_api': + listening_service => 'midonet_api', + ports => '8081', + ipaddresses => hiera('midonet_api_node_ips', $controller_hosts_real), + server_names => $controller_hosts_names_real, + options => ['check', 'inter 2000', 'rise 2', 'fall 5'], + } + } } diff --git a/tripleo/manifests/network/midonet/agent.pp b/tripleo/manifests/network/midonet/agent.pp new file mode 100644 index 000000000..2f1da9414 --- /dev/null +++ b/tripleo/manifests/network/midonet/agent.pp @@ -0,0 +1,61 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# == Class: tripleo::network::midonet::agent +# +# Configure the midonet agent +# +# == Parameters: +# +# [*zookeeper_servers*] +# (required) List of IPs of the zookeeper server cluster. It will configure +# the connection using the 2181 port. +# Array of strings value. +# +# [*cassandra_seeds*] +# (required) List of IPs of the cassandra cluster. +# Array of strings value. +# +class tripleo::network::midonet::agent ( + $zookeeper_servers, + $cassandra_seeds +) { + + validate_array($zookeeper_servers) + validate_array($cassandra_seeds) + + # FIXME: This statement should be controlled by hiera on heat templates + # project + # Make sure openvswitch service is not running + service {'openvswitch': + ensure => stopped, + enable => false + } + + exec {'delete datapaths': + command => '/usr/bin/mm-dpctl --delete-dp ovs-system', + path => '/usr/bin:/usr/sbin:/bin', + onlyif => '/usr/bin/mm-dpctl --show-dp ovs-system' + } + + # Configure and run the agent + class {'::midonet::midonet_agent': + zk_servers => list_to_zookeeper_hash($zookeeper_servers), + cassandra_seeds => $cassandra_seeds + } + + Service['openvswitch'] -> Class['::midonet::midonet_agent::run'] + Exec['delete datapaths'] -> Class['::midonet::midonet_agent::run'] +} diff --git a/tripleo/manifests/network/midonet/api.pp b/tripleo/manifests/network/midonet/api.pp new file mode 100644 index 000000000..33b22178a --- /dev/null +++ b/tripleo/manifests/network/midonet/api.pp @@ -0,0 +1,117 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# == Class: tripleo::network::midonet::api +# +# Configure the MidoNet API +# +# == Parameters: +# +# [*zookeeper_servers*] +# (required) List IPs of the zookeeper server cluster. Zookeeper is the +# backend database where MidoNet stores the virtual network topology. +# Array of strings value. +# +# [*vip*] +# (required) Public Virtual IP where the API will be exposed. +# String (IPv4) value. +# +# [*keystone_ip*] +# (required) MidoNet API is registered as an OpenStack service. Provide the +# keystone ip address. +# String (IPv4) value. +# +# [*keystone_admin_token*] +# (required) MidoNet API is registered as an OpenStack service. It needs the +# keystone admin token to perform some admin calls. +# String value. +# +# [*bind_address*] +# (required) MidoNet API uses a Tomcat instance to offer the REST service. The +# ip address where to bind the tomcat service. +# String (IPv4) value. +# +# [*admin_password*] +# (required) OpenStack admin user password. +# String value. +# +# [*keystone_port*] +# (optional) MidoNet API is registered as an OpenStack service. Provide +# the keystone port. +# Defaults to 35357 +# +# [*keystone_tenant_name*] +# (optional) Tenant of the keystone service. +# Defaults to 'admin' +# +# [*admin_user_name*] +# (optional) OpenStack admin user name. +# Defaults to 'admin' +# +# [*admin_tenant_name*] +# (optional). OpenStack admin tenant name. +# Defaults to 'admin' +# + +class tripleo::network::midonet::api( + $zookeeper_servers, + $vip, + $keystone_ip, + $keystone_admin_token, + $bind_address, + $admin_password, + $keystone_port = 35357, + $keystone_tenant_name = 'admin', + $admin_user_name = 'admin', + $admin_tenant_name = 'admin' +) +{ + validate_array($zookeeper_servers) + validate_ip_address($vip) + validate_ip_address($keystone_ip) + validate_ip_address($bind_address) + + # Run Tomcat and MidoNet API + class {'::tomcat': + install_from_source => false + } -> + + package {'midonet-api': + ensure => present + } -> + + class {'::midonet::midonet_api::run': + zk_servers => list_to_zookeeper_hash($zookeeper_servers), + keystone_auth => true, + tomcat_package => 'tomcat', + vtep => false, + api_ip => $vip, + api_port => '8081', + keystone_host => $keystone_ip, + keystone_port => $keystone_port, + keystone_admin_token => $keystone_admin_token, + keystone_tenant_name => $keystone_tenant_name, + catalina_base => '/usr/share/tomcat', + bind_address => $bind_address + } + + # Configure the CLI + class {'::midonet::midonet_cli': + api_endpoint => "http://${vip}:8081/midonet-api", + username => $admin_user_name, + password => $admin_password, + tenant_name => $admin_tenant_name + } +} diff --git a/tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb b/tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb new file mode 100644 index 000000000..13be98ecb --- /dev/null +++ b/tripleo/spec/classes/tripleo_cluster_cassandra_spec.rb @@ -0,0 +1,54 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# Unit tests for the cassandra service + +require 'spec_helper' + +describe 'tripleo::cluster::cassandra' do + + shared_examples_for 'cassandra cluster service' do + + let :facts do + { + :hostname => 'host1.midonet', + :osfamily => 'RedHat', + :operatingsystemmajrelease => 7, + } + end + + let :params do + { + :cassandra_servers => ['192.168.2.2', '192.168.2.3'], + :cassandra_ip => '192.168.2.2' + } + end + + it 'should configure cassandra' do + is_expected.to contain_class('cassandra').with( + :seeds => ['192.168.2.2', '192.168.2.3'], + :listen_address => '192.168.2.2', + :storage_port => 7000, + :ssl_storage_port => 7001, + :native_transport_port => 9042, + :rpc_port => 9160 + ) + + end + end + + it_configures 'cassandra cluster service' + +end diff --git a/tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb b/tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb new file mode 100644 index 000000000..ed46164a4 --- /dev/null +++ b/tripleo/spec/classes/tripleo_cluster_zookeeper_spec.rb @@ -0,0 +1,115 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# Unit tests for the zookeeper service + +require 'spec_helper' + +describe 'tripleo::cluster::zookeeper' do + + let :default_params do + { + :zookeeper_server_ips => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], + :zookeeper_hostnames => ['host1.midonet', 'host2.midonet', 'host3.midonet'] + } + end + + context 'on host1' do + let :facts do + { + :hostname => 'host1.midonet', + :osfamily => 'RedHat', + :operatingsystemmajrelease => 7, + } + end + + let :params do + { + :zookeeper_client_ip => '23.43.2.34' + } + end + + before do + params.merge!(default_params) + end + + it 'should call zookeeper using id==1' do + is_expected.to contain_class('zookeeper').with( + :servers => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], + :client_ip => '23.43.2.34', + :id => 1 + ) + end + + end + + context 'on host2' do + let :facts do + { + :hostname => 'host2.midonet', + :osfamily => 'RedHat', + :operatingsystemmajrelease => 7, + } + end + + let :params do + { + :zookeeper_client_ip => '23.43.2.35' + } + end + + before do + params.merge!(default_params) + end + + it 'should call zookeeper using id==1' do + is_expected.to contain_class('zookeeper').with( + :servers => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], + :client_ip => '23.43.2.35', + :id => 2 + ) + end + end + + context 'on host3' do + let :facts do + { + :hostname => 'host3.midonet', + :osfamily => 'RedHat', + :operatingsystemmajrelease => 7, + } + end + + let :params do + { + :zookeeper_client_ip => '23.43.2.36' + } + end + + before do + params.merge!(default_params) + end + + it 'should call zookeeper using id==1' do + is_expected.to contain_class('zookeeper').with( + :servers => ['23.43.2.34', '23.43.2.35', '24.43.2.36'], + :client_ip => '23.43.2.36', + :id => 3 + ) + end + + end + +end diff --git a/tripleo/spec/classes/tripleo_firewall_spec.rb b/tripleo/spec/classes/tripleo_firewall_spec.rb index c1249b9fa..7d1d1ecb8 100644 --- a/tripleo/spec/classes/tripleo_firewall_spec.rb +++ b/tripleo/spec/classes/tripleo_firewall_spec.rb @@ -105,7 +105,10 @@ context 'on RedHat platforms' do let :facts do - { :osfamily => 'RedHat' } + { + :osfamily => 'RedHat', + :operatingsystemrelease => '7.1', + } end it_configures 'tripleo node' diff --git a/tripleo/spec/classes/tripleo_midonet_agent_spec.rb b/tripleo/spec/classes/tripleo_midonet_agent_spec.rb new file mode 100644 index 000000000..eb9195e06 --- /dev/null +++ b/tripleo/spec/classes/tripleo_midonet_agent_spec.rb @@ -0,0 +1,61 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# Unit tests for the midonet agent + +require 'spec_helper' + +describe 'tripleo::network::midonet::agent' do + + let :facts do + { + :hostname => 'host2.midonet', + :osfamily => 'RedHat', + :operatingsystem => 'CentOS', + :operatingsystemmajrelease => 7, + } + end + + shared_examples_for 'midonet agent test' do + + let :params do + { + :zookeeper_servers => ['192.168.2.2', '192.168.2.3'], + :cassandra_seeds => ['192.168.2.2', '192.168.2.3'] + } + end + + it 'should stop openvswitch' do + is_expected.to contain_service('openvswitch').with( + :ensure => 'stopped', + :enable => false + ) + end + + it 'should run the agent with a list of maps' do + is_expected.to contain_class('midonet::midonet_agent').with( + :zk_servers => [{'ip' => '192.168.2.2', + 'port' => 2181}, + {'ip' => '192.168.2.3', + 'port' => 2181}], + :cassandra_seeds => ['192.168.2.2','192.168.2.3'] + ) + end + end + + it_configures 'midonet agent test' + + +end diff --git a/tripleo/spec/classes/tripleo_midonet_api_spec.rb b/tripleo/spec/classes/tripleo_midonet_api_spec.rb new file mode 100644 index 000000000..4b4729494 --- /dev/null +++ b/tripleo/spec/classes/tripleo_midonet_api_spec.rb @@ -0,0 +1,72 @@ +# +# Copyright (C) 2015 Midokura SARL +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# Unit tests for the midonet api + +require 'spec_helper' + +describe 'tripleo::network::midonet::api' do + + let :facts do + { + :augeasversion => '1.0.0' + } + end + + shared_examples_for 'midonet api test' do + + let :params do + { + :zookeeper_servers => ['192.168.2.1', '192.168.2.2'], + :vip => '192.23.0.2', + :keystone_ip => '192.23.0.2', + :keystone_admin_token => 'admin_token', + :admin_password => 'admin_password', + :bind_address => '192.23.0.65' + } + end + + it 'should call api configuration' do + is_expected.to contain_class('midonet::midonet_api::run').with( + :zk_servers => [{'ip' => '192.168.2.1', 'port' => 2181}, + {'ip' => '192.168.2.2', 'port' => 2181}], + :keystone_auth => true, + :tomcat_package => 'tomcat', + :vtep => false, + :api_ip => '192.23.0.2', + :api_port => '8081', + :keystone_host => '192.23.0.2', + :keystone_port => 35357, + :keystone_admin_token => 'admin_token', + :keystone_tenant_name => 'admin', + :catalina_base => '/usr/share/tomcat', + :bind_address => '192.23.0.65' + ) + end + + it 'should install the cli' do + is_expected.to contain_class('midonet::midonet_cli').with( + :api_endpoint => 'http://192.23.0.2:8081/midonet-api', + :username => 'admin', + :password => 'admin_password', + :tenant_name => 'admin' + ) + end + + end + + it_configures 'midonet api test' + +end diff --git a/tripleo/spec/spec_helper.rb b/tripleo/spec/spec_helper.rb index 700be6a27..5cf9642eb 100644 --- a/tripleo/spec/spec_helper.rb +++ b/tripleo/spec/spec_helper.rb @@ -2,9 +2,13 @@ require 'shared_examples' require 'webmock/rspec' +fixture_path = File.expand_path(File.join(__FILE__, '..', 'fixtures')) + RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' c.alias_it_should_behave_like_to :it_raises, 'raises' + c.module_path = File.join(fixture_path, 'modules') + c.manifest_dir = File.join(fixture_path, 'manifests') c.default_facts = { :kernel => 'Linux', diff --git a/trove/manifests/keystone/auth.pp b/trove/manifests/keystone/auth.pp index 7155b8d39..c18d842ff 100644 --- a/trove/manifests/keystone/auth.pp +++ b/trove/manifests/keystone/auth.pp @@ -202,7 +202,7 @@ Keystone_user_role["${auth_name}@${tenant}"] ~> Service <| name == 'trove-server' |> - Keystone_endpoint<| title == "${region}/${real_service_name}" |> + Keystone_endpoint<| title == "${region}/${real_service_name}::${service_type}" |> ~> Service <| name == 'trove-server' |> keystone::resource::service_identity { 'trove': diff --git a/trove/manifests/taskmanager.pp b/trove/manifests/taskmanager.pp index cb42167f8..2c7cc0437 100644 --- a/trove/manifests/taskmanager.pp +++ b/trove/manifests/taskmanager.pp @@ -304,7 +304,8 @@ if $guestagent_config_file { if $use_guestagent_template { file { $guestagent_config_file: - content => template('trove/trove-guestagent.conf.erb') + content => template('trove/trove-guestagent.conf.erb'), + require => Package[$::trove::params::taskmanager_package_name] } } else { class {'::trove::guestagent': diff --git a/trove/spec/classes/trove_keystone_auth_spec.rb b/trove/spec/classes/trove_keystone_auth_spec.rb index ac2ea14ea..4b26c5e8b 100644 --- a/trove/spec/classes/trove_keystone_auth_spec.rb +++ b/trove/spec/classes/trove_keystone_auth_spec.rb @@ -42,13 +42,12 @@ :roles => ['admin'] )} - it { is_expected.to contain_keystone_service('trove').with( + it { is_expected.to contain_keystone_service('trove::database').with( :ensure => 'present', - :type => 'database', :description => 'Trove Database Service' ) } - it { is_expected.to contain_keystone_endpoint('RegionOne/trove').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/trove::database').with( :ensure => 'present', :public_url => "http://127.0.0.1:8779/v1.0/%(tenant_id)s", :admin_url => "http://127.0.0.1:8779/v1.0/%(tenant_id)s", @@ -75,7 +74,7 @@ :admin_url => 'http://10.10.10.12:81/v1.0/%(tenant_id)s' } end - it { is_expected.to contain_keystone_endpoint('RegionOne/trove').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/trove::database').with( :ensure => 'present', :public_url => 'https://10.10.10.10:80/v1.0/%(tenant_id)s', :internal_url => 'http://10.10.10.11:81/v1.0/%(tenant_id)s', @@ -96,7 +95,7 @@ :admin_address => '10.10.10.12' } end - it { is_expected.to contain_keystone_endpoint('RegionOne/trove').with( + it { is_expected.to contain_keystone_endpoint('RegionOne/trove::database').with( :ensure => 'present', :public_url => 'https://10.10.10.10:80/v1.0/%(tenant_id)s', :internal_url => 'https://10.10.10.11:81/v1.0/%(tenant_id)s', @@ -112,8 +111,8 @@ it { is_expected.to contain_keystone_user('trovey') } it { is_expected.to contain_keystone_user_role('trovey@services') } - it { is_expected.to contain_keystone_service('trovey') } - it { is_expected.to contain_keystone_endpoint('RegionOne/trovey') } + it { is_expected.to contain_keystone_service('trovey::database') } + it { is_expected.to contain_keystone_endpoint('RegionOne/trovey::database') } end describe 'when overriding service name' do @@ -125,8 +124,8 @@ it { is_expected.to contain_keystone_user('trove') } it { is_expected.to contain_keystone_user_role('trove@services') } - it { is_expected.to contain_keystone_service('trove_service') } - it { is_expected.to contain_keystone_endpoint('RegionOne/trove_service') } + it { is_expected.to contain_keystone_service('trove_service::database') } + it { is_expected.to contain_keystone_endpoint('RegionOne/trove_service::database') } end describe 'when disabling endpoint configuration' do @@ -135,6 +134,6 @@ :password => 'trove_password' } end - it { is_expected.to_not contain_keystone_endpoint('RegionOne/neutron') } + it { is_expected.to_not contain_keystone_endpoint('RegionOne/neutron::database') } end end From cff3e7fca85df69865fd57532c322fcd1585b7aa Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Fri, 15 Jan 2016 16:10:27 -0500 Subject: [PATCH 14/35] Update pacemaker to 00d99b66931d520af202b94d0a2fe3b43f11d8ee 00d99b66931d520af202b94d0a2fe3b43f11d8ee Merge pull request #70 from sathlan/rhbz/1298716 4ae9fabb3ff50dfc95a5f6b415e9e914a6ae8b00 Resource constraint with colon in the name (IPv6) Change-Id: I67791d7cc138c2e99b6b966f13c89c548df34f3c --- Puppetfile | 2 +- .../provider/pcmk_constraint/default.rb | 27 ++++++++++++------- pacemaker/manifests/constraint/base.pp | 24 ++++++++++------- 3 files changed, 33 insertions(+), 20 deletions(-) diff --git a/Puppetfile b/Puppetfile index 0a0a3fe0b..10d35337e 100644 --- a/Puppetfile +++ b/Puppetfile @@ -191,7 +191,7 @@ mod 'openstacklib', :git => 'https://github.com/openstack/puppet-openstacklib.git' mod 'pacemaker', - :commit => 'e9a8f0d048c1bbc86a4bb1ee81417da3afe6673e', + :commit => '00d99b66931d520af202b94d0a2fe3b43f11d8ee', :git => 'https://github.com/redhat-openstack/puppet-pacemaker.git' mod 'puppet', diff --git a/pacemaker/lib/puppet/provider/pcmk_constraint/default.rb b/pacemaker/lib/puppet/provider/pcmk_constraint/default.rb index 1c7c426c9..ae397b9c5 100644 --- a/pacemaker/lib/puppet/provider/pcmk_constraint/default.rb +++ b/pacemaker/lib/puppet/provider/pcmk_constraint/default.rb @@ -3,14 +3,18 @@ ### overloaded methods def create + resource_resource = @resource[:resource].gsub(':', '.') + resource_name = @resource[:name].gsub(':', '.') + resource_location = @resource[:location].gsub(':', '.') case @resource[:constraint_type] when :location - cmd = 'constraint location add ' + @resource[:name] + ' ' + @resource[:resource] + ' ' + @resource[:location] + ' ' + @resource[:score] + cmd = 'constraint location add ' + resource_name + ' ' + resource_resource + ' ' + @resource[:location] + ' ' + @resource[:score] when :colocation + resource_location = @resource[:location].gsub(':', '.') if @resource[:master_slave] - cmd = 'constraint colocation add ' + @resource[:resource] + ' with master ' + @resource[:location] + ' ' + @resource[:score] + cmd = 'constraint colocation add ' + resource_resource + ' with master ' + resource_location + ' ' + @resource[:score] else - cmd = 'constraint colocation add ' + @resource[:resource] + ' with ' + @resource[:location] + ' ' + @resource[:score] + cmd = 'constraint colocation add ' + resource_resource + ' with ' + resource_location + ' ' + @resource[:score] end else fail(String(@resource[:constraint_type]) + ' is an invalid location type') @@ -21,30 +25,35 @@ def create end def destroy + resource_resource = @resource[:resource].gsub(':', '.') + resource_name = @resource[:name].gsub(':', '.') + resource_location = @resource[:location].gsub(':', '.') case @resource[:constraint_type] when :location - cmd = 'constraint location remove ' + @resource[:name] + cmd = 'constraint location remove ' + resource_name when :colocation - cmd = 'constraint colocation remove ' + @resource[:resource] + ' ' + @resource[:location] + cmd = 'constraint colocation remove ' + resource_resource + ' ' + resource_location end pcs('constraint delete', cmd) end def exists? + resource_name = @resource[:name].gsub(':', '.') + resource_resource = @resource[:resource].gsub(':', '.') + resource_location = @resource[:location].gsub(':', '.') cmd = 'constraint ' + String(@resource[:constraint_type]) + ' show --full' pcs_out = pcs('show', cmd) - # find the constraint for line in pcs_out.lines.each do case @resource[:constraint_type] when :location - return true if line.include? @resource[:name] + return true if line.include? resource_name when :colocation if @resource[:master_slave] - return true if line.include? @resource[:resource] + ' with ' + @resource[:location] and line.include? "with-rsc-role:Master" + return true if line.include? resource_resource + ' with ' + resource_location and line.include? "with-rsc-role:Master" else - return true if line.include? @resource[:resource] + ' with ' + @resource[:location] + return true if line.include? resource_resource + ' with ' + resource_location end end end diff --git a/pacemaker/manifests/constraint/base.pp b/pacemaker/manifests/constraint/base.pp index c58e26412..a9514580f 100644 --- a/pacemaker/manifests/constraint/base.pp +++ b/pacemaker/manifests/constraint/base.pp @@ -32,19 +32,23 @@ $_constraint_params = "" } + $first_resource_cleaned = regsubst($first_resource, '(:)', '.', 'G') + $second_resource_cleaned = regsubst($second_resource, '(:)', '.', 'G') + if($ensure == absent) { if($constraint_type == 'location') { + $name_cleaned = regsubst($name, '(:)', '.', 'G') exec { "Removing location constraint ${name}": - command => "/usr/sbin/pcs constraint location remove ${name}", - onlyif => "/usr/sbin/pcs constraint location show --full | grep ${name}", + command => "/usr/sbin/pcs constraint location remove ${name_cleaned}", + onlyif => "/usr/sbin/pcs constraint location show --full | grep ${name_cleaned}", require => Exec["wait-for-settle"], tries => $tries, try_sleep => $try_sleep, } } else { exec { "Removing ${constraint_type} constraint ${name}": - command => "/usr/sbin/pcs constraint ${constraint_type} remove ${first_resource} ${second_resource}", - onlyif => "/usr/sbin/pcs constraint ${constraint_type} show | grep ${first_resource} | grep ${second_resource}", + command => "/usr/sbin/pcs constraint ${constraint_type} remove ${first_resource_cleaned} ${second_resource_cleaned}", + onlyif => "/usr/sbin/pcs constraint ${constraint_type} show | grep ${first_resource_cleaned} | grep ${second_resource_cleaned}", require => Exec["wait-for-settle"], tries => $tries, try_sleep => $try_sleep, @@ -55,8 +59,8 @@ 'colocation': { fail("Deprecated use pacemaker::constraint::colocation") exec { "Creating colocation constraint ${name}": - command => "/usr/sbin/pcs constraint colocation add ${first_resource} ${second_resource} ${score}", - unless => "/usr/sbin/pcs constraint colocation show | grep ${first_resource} | grep ${second_resource} > /dev/null 2>&1", + command => "/usr/sbin/pcs constraint colocation add ${first_resource_cleaned} ${second_resource_cleaned} ${score}", + unless => "/usr/sbin/pcs constraint colocation show | grep ${first_resource_cleaned} | grep ${second_resource_cleaned} > /dev/null 2>&1", require => [Exec["wait-for-settle"],Package["pcs"]], tries => $tries, try_sleep => $try_sleep, @@ -64,8 +68,8 @@ } 'order': { exec { "Creating order constraint ${name}": - command => "/usr/sbin/pcs constraint order ${first_action} ${first_resource} then ${second_action} ${second_resource} ${_constraint_params}", - unless => "/usr/sbin/pcs constraint order show | grep ${first_resource} | grep ${second_resource} > /dev/null 2>&1", + command => "/usr/sbin/pcs constraint order ${first_action} ${first_resource_cleaned} then ${second_action} ${second_resource_cleaned} ${_constraint_params}", + unless => "/usr/sbin/pcs constraint order show | grep ${first_resource_cleaned} | grep ${second_resource_cleaned} > /dev/null 2>&1", require => [Exec["wait-for-settle"],Package["pcs"]], tries => $tries, try_sleep => $try_sleep, @@ -74,8 +78,8 @@ 'location': { fail("Deprecated use pacemaker::constraint::location") exec { "Creating location constraint ${name}": - command => "/usr/sbin/pcs constraint location add ${name} ${first_resource} ${location} ${score}", - unless => "/usr/sbin/pcs constraint location show | grep ${first_resource} > /dev/null 2>&1", + command => "/usr/sbin/pcs constraint location add ${name} ${first_resource_cleaned} ${location} ${score}", + unless => "/usr/sbin/pcs constraint location show | grep ${first_resource_cleaned} > /dev/null 2>&1", require => [Exec["wait-for-settle"],Package["pcs"]], tries => $tries, try_sleep => $try_sleep, From 09513f7edefd2599986dd64e5061489ce7913c10 Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Fri, 15 Jan 2016 16:16:26 -0500 Subject: [PATCH 15/35] Sync openstack puppet modules to stable/liberty upstream. Update glance to 66abdfea280a68edae92c291e194df542b5f64da 66abdfea280a68edae92c291e194df542b5f64da Fix Swift store configuration Change-Id: I0520e8083f93c373fe168ca459b69811808cabfe Update ironic to d97c722fa3c147b9ab45f4837dc5ced4ea642d6b d97c722fa3c147b9ab45f4837dc5ced4ea642d6b Enable changing list of inspection collectors c0dd0b65fe197e0ae45a28e37eb0bd7643c40d06 [inspector] Specify SELinux type context for tftp dir Change-Id: I299860cff6affd42b69f8d70c7173ee534e8487c Update neutron to ade5a77e9be344234ade6267e3beca4cd2556b43 ade5a77e9be344234ade6267e3beca4cd2556b43 Default qos_notification_drivers to upstream 5f024d9d395c5321492c69085df6de09bf3c2c37 Adds the ability to force metadata with dhcp agent Change-Id: I22e4ed385072da40189b41431f02d29f0d584211 Update nova to 2ec3e65e34a35e1099e07b79546e5b4de43c014a 2ec3e65e34a35e1099e07b79546e5b4de43c014a Merge "Use auth_uri to get nova credentials" into stable/liberty d80f7a25d6ceb9390847ae77090b52663ef36dfb Allow to enable/disable IPv6 support b30e85f8391419df46040aed94971f9ceacd6759 Use auth_uri to get nova credentials Change-Id: Icdf024601f5e6874af7b47b497a7476e419b0bab Update trove to 78eaf20cb21e6ff29569701847b6d8d448e9dfe5 78eaf20cb21e6ff29569701847b6d8d448e9dfe5 Follow-up on PyMySQL support for Red Hat platforms 04036ce46fcafc112c38b4492d1e5d9777b6b379 Support of PyMySQL driver for MySQL backend Change-Id: I91c7300733aaf9f949f291b052a63abeefdc310e Update vswitch to 0abd9227ed509e5aa63aa52eba991e6bee13f67e 0abd9227ed509e5aa63aa52eba991e6bee13f67e Remove duplicate loading of constants Change-Id: I701e4143741597ba63239ffec88b56e646ff3238 --- Puppetfile | 12 +++--- glance/manifests/backend/swift.pp | 2 +- .../spec/classes/glance_backend_swift_spec.rb | 2 +- ironic/manifests/inspector.pp | 12 ++++-- ironic/spec/classes/ironic_inspector_spec.rb | 12 ++++++ ironic/templates/inspector_ipxe.erb | 2 +- ironic/templates/inspector_pxelinux_cfg.erb | 2 +- neutron/manifests/agents/dhcp.pp | 10 ++++- neutron/manifests/server.pp | 4 +- .../spec/classes/neutron_agents_dhcp_spec.rb | 28 +++++++++++-- neutron/spec/classes/neutron_server_spec.rb | 2 +- nova/lib/puppet/provider/nova.rb | 6 +-- nova/manifests/init.pp | 7 +++- nova/spec/classes/nova_init_spec.rb | 3 ++ nova/spec/unit/provider/nova_spec.rb | 6 +-- trove/manifests/db.pp | 12 ++++-- trove/manifests/params.pp | 2 + trove/spec/acceptance/basic_trove_spec.rb | 2 +- trove/spec/classes/trove_db_spec.rb | 41 ++++++++++++++++++- .../lib/puppet/provider/vs_port/ovs_redhat.rb | 21 +++++----- 20 files changed, 143 insertions(+), 45 deletions(-) diff --git a/Puppetfile b/Puppetfile index 10d35337e..895825747 100644 --- a/Puppetfile +++ b/Puppetfile @@ -71,7 +71,7 @@ mod 'git', :git => 'https://github.com/puppetlabs/puppetlabs-git.git' mod 'glance', - :commit => '4c7c407fbd15977d81ca4df6309ad1a839ad59e1', + :commit => '66abdfea280a68edae92c291e194df542b5f64da', :git => 'https://github.com/openstack/puppet-glance.git' mod 'gluster', @@ -103,7 +103,7 @@ mod 'ipa', :git => 'https://github.com/xbezdick/puppet-ipa-1.git' mod 'ironic', - :commit => '342cc10fb080a787d971f39df0901750996807d3', + :commit => 'd97c722fa3c147b9ab45f4837dc5ced4ea642d6b', :git => 'https://github.com/openstack/puppet-ironic.git' mod 'java', @@ -163,11 +163,11 @@ mod 'nagios', :git => 'https://github.com/gildub/puppet-nagios-openstack.git' mod 'neutron', - :commit => '3908534c2d05a5040f6adf436f8f96a22be528bb', + :commit => 'ade5a77e9be344234ade6267e3beca4cd2556b43', :git => 'https://github.com/openstack/puppet-neutron.git' mod 'nova', - :commit => '972c4d6e76a539c59b2df78b25ca03859f2144bb', + :commit => '2ec3e65e34a35e1099e07b79546e5b4de43c014a', :git => 'https://github.com/openstack/puppet-nova.git' mod 'nssdb', @@ -267,7 +267,7 @@ mod 'tripleo', :git => 'https://github.com/openstack/puppet-tripleo.git' mod 'trove', - :commit => 'e2e6b767f82989ce6e38b77c2f33b6fe7efe1b6d', + :commit => '78eaf20cb21e6ff29569701847b6d8d448e9dfe5', :git => 'https://github.com/openstack/puppet-trove' mod 'tuskar', @@ -287,7 +287,7 @@ mod 'vlan', :git => 'https://github.com/derekhiggins/puppet-vlan.git' mod 'vswitch', - :commit => 'c374840910c823f7669cf2e1229c7df7192ae880', + :commit => '0abd9227ed509e5aa63aa52eba991e6bee13f67e', :git => 'https://github.com/openstack/puppet-vswitch.git' mod 'xinetd', diff --git a/glance/manifests/backend/swift.pp b/glance/manifests/backend/swift.pp index 56f58051b..3a1adda4c 100644 --- a/glance/manifests/backend/swift.pp +++ b/glance/manifests/backend/swift.pp @@ -62,7 +62,7 @@ # == class: glance::backend::swift 'glance_store/swift_store_endpoint_type': value => $swift_store_endpoint_type; - 'DEFAULT/swift_store_config_file': value => '/etc/glance/glance-api.conf'; + 'glance_store/swift_store_config_file': value => '/etc/glance/glance-api.conf'; 'glance_store/default_swift_reference': value => $default_swift_reference; "${default_swift_reference}/user": value => $swift_store_user; "${default_swift_reference}/key": value => $swift_store_key; diff --git a/glance/spec/classes/glance_backend_swift_spec.rb b/glance/spec/classes/glance_backend_swift_spec.rb index 3922cfc6e..7ca3fa32c 100644 --- a/glance/spec/classes/glance_backend_swift_spec.rb +++ b/glance/spec/classes/glance_backend_swift_spec.rb @@ -27,7 +27,7 @@ is_expected.to contain_glance_api_config('glance_store/swift_store_create_container_on_put').with_value(false) is_expected.to contain_glance_api_config('glance_store/swift_store_endpoint_type').with_value('internalURL') is_expected.to contain_glance_api_config('glance_store/swift_store_region').with_value(nil) - is_expected.to contain_glance_api_config('DEFAULT/swift_store_config_file').with_value('/etc/glance/glance-api.conf') + is_expected.to contain_glance_api_config('glance_store/swift_store_config_file').with_value('/etc/glance/glance-api.conf') is_expected.to contain_glance_api_config('glance_store/default_swift_reference').with_value('ref1') is_expected.to contain_glance_api_config('ref1/key').with_value('key') is_expected.to contain_glance_api_config('ref1/user').with_value('user') diff --git a/ironic/manifests/inspector.pp b/ironic/manifests/inspector.pp index 773a79d6a..dc7829e3b 100644 --- a/ironic/manifests/inspector.pp +++ b/ironic/manifests/inspector.pp @@ -130,6 +130,10 @@ # Enable dbsync # Defaults to true # +# [*ramdisk_collectors*] +# Comma-separated list of IPA inspection collectors +# Defaults to 'default' +# class ironic::inspector ( $package_ensure = 'present', $enabled = true, @@ -159,6 +163,7 @@ $dnsmasq_ip_range = '192.168.0.100,192.168.0.120', $dnsmasq_local_ip = '192.168.0.1', $sync_db = true, + $ramdisk_collectors = 'default', ) { include ::ironic::params @@ -169,6 +174,10 @@ ensure => 'present', require => Package['ironic-inspector'], } + file { '/tftpboot': + ensure => 'directory', + seltype => 'tftpdir_t', + } if $pxe_transfer_protocol == 'tftp' { file { '/etc/ironic-inspector/dnsmasq.conf': @@ -176,9 +185,6 @@ content => template('ironic/inspector_dnsmasq_tftp.erb'), require => Package['ironic-inspector'], } - file { '/tftpboot': - ensure => 'directory', - } file { '/tftpboot/pxelinux.cfg': ensure => 'directory', } diff --git a/ironic/spec/classes/ironic_inspector_spec.rb b/ironic/spec/classes/ironic_inspector_spec.rb index ea8cb4cb5..9013a2d9f 100644 --- a/ironic/spec/classes/ironic_inspector_spec.rb +++ b/ironic/spec/classes/ironic_inspector_spec.rb @@ -118,6 +118,15 @@ 'require' => 'Package[ironic-inspector]', 'content' => /default/, ) + is_expected.to contain_file('/tftpboot/pxelinux.cfg/default').with_content( + /initrd=agent.ramdisk ipa-inspection-callback-url=http:\/\/192.168.0.1:5050\/v1\/continue ipa-inspection-collectors=default/ + ) + end + it 'should contain directory /tftpboot with selinux type tftpdir_t' do + is_expected.to contain_file('/tftpboot').with( + 'ensure' => 'directory', + 'seltype' => 'tftpdir_t' + ) end context 'when overriding parameters' do @@ -158,6 +167,9 @@ 'require' => 'Package[ironic-inspector]', 'content' => /ipxe/, ) + is_expected.to contain_file('/httpboot/inspector.ipxe').with_content( + /kernel http:\/\/192.168.0.1:8088\/agent.kernel ipa-inspection-callback-url=http:\/\/192.168.0.1:5050\/v1\/continue ipa-inspection-collectors=default/ + ) end end end diff --git a/ironic/templates/inspector_ipxe.erb b/ironic/templates/inspector_ipxe.erb index d5129f30d..957626e88 100644 --- a/ironic/templates/inspector_ipxe.erb +++ b/ironic/templates/inspector_ipxe.erb @@ -2,6 +2,6 @@ dhcp -kernel http://<%= @dnsmasq_local_ip %>:8088/agent.kernel ipa-inspection-callback-url=http://<%= @dnsmasq_local_ip %>:5050/v1/continue systemd.journald.forward_to_console=yes BOOTIF=${mac} +kernel http://<%= @dnsmasq_local_ip %>:8088/agent.kernel ipa-inspection-callback-url=http://<%= @dnsmasq_local_ip %>:5050/v1/continue ipa-inspection-collectors=<%= @ramdisk_collectors %> systemd.journald.forward_to_console=yes BOOTIF=${mac} initrd http://<%= @dnsmasq_local_ip %>:8088/agent.ramdisk boot diff --git a/ironic/templates/inspector_pxelinux_cfg.erb b/ironic/templates/inspector_pxelinux_cfg.erb index f25c4c7c5..8730ea7e3 100644 --- a/ironic/templates/inspector_pxelinux_cfg.erb +++ b/ironic/templates/inspector_pxelinux_cfg.erb @@ -2,5 +2,5 @@ default inspector label inspector kernel agent.kernel -append initrd=agent.ramdisk ipa-inspection-callback-url=http://<%= @dnsmasq_local_ip %>:5050/v1/continue systemd.journald.forward_to_console=yes +append initrd=agent.ramdisk ipa-inspection-callback-url=http://<%= @dnsmasq_local_ip %>:5050/v1/continue ipa-inspection-collectors=<%= @ramdisk_collectors %> systemd.journald.forward_to_console=yes ipappend 3 diff --git a/neutron/manifests/agents/dhcp.pp b/neutron/manifests/agents/dhcp.pp index 0c1049205..cf772b54d 100644 --- a/neutron/manifests/agents/dhcp.pp +++ b/neutron/manifests/agents/dhcp.pp @@ -53,6 +53,10 @@ # (optional) enable metadata support on isolated networks. # Defaults to false. # +# [*enable_force_metadata*] +# (optional) enable metadata support on all networks. +# Defaults to false. +# # [*enable_metadata_network*] # (optional) Allows for serving metadata requests coming from a dedicated metadata # access network whose cidr is 169.254.169.254/16 (or larger prefix), and is @@ -86,6 +90,7 @@ $dnsmasq_config_file = undef, $dhcp_delete_namespaces = true, $enable_isolated_metadata = false, + $enable_force_metadata = false, $enable_metadata_network = false, $dhcp_broadcast_reply = false, # DEPRECATED PARAMETERS @@ -110,11 +115,12 @@ } } - if (! $enable_isolated_metadata) and $enable_metadata_network { - fail('enable_metadata_network to true requires enable_isolated_metadata also enabled.') + if (! ($enable_isolated_metadata or $enable_force_metadata)) and $enable_metadata_network { + fail('enable_metadata_network to true requires enable_isolated_metadata or enable_force_metadata also enabled.') } else { neutron_dhcp_agent_config { 'DEFAULT/enable_isolated_metadata': value => $enable_isolated_metadata; + 'DEFAULT/force_metadata': value => $enable_force_metadata; 'DEFAULT/enable_metadata_network': value => $enable_metadata_network; } } diff --git a/neutron/manifests/server.pp b/neutron/manifests/server.pp index f44ffd758..5f317541e 100644 --- a/neutron/manifests/server.pp +++ b/neutron/manifests/server.pp @@ -198,7 +198,7 @@ # # [*qos_notification_drivers*] # (optional) Drivers list to use to send the update notification -# Defaults to []. +# Defaults to ['message_queue']. # class neutron::server ( $package_ensure = 'present', @@ -230,7 +230,7 @@ $max_l3_agents_per_router = 3, $min_l3_agents_per_router = 2, $l3_ha_net_cidr = '169.254.192.0/18', - $qos_notification_drivers = [], + $qos_notification_drivers = ['message_queue'], # DEPRECATED PARAMETERS $auth_host = 'localhost', $auth_port = '35357', diff --git a/neutron/spec/classes/neutron_agents_dhcp_spec.rb b/neutron/spec/classes/neutron_agents_dhcp_spec.rb index a06f47295..23b05c5d6 100644 --- a/neutron/spec/classes/neutron_agents_dhcp_spec.rb +++ b/neutron/spec/classes/neutron_agents_dhcp_spec.rb @@ -25,6 +25,7 @@ :dhcp_delete_namespaces => true, :enable_isolated_metadata => false, :enable_metadata_network => false, + :enable_force_metadata => false, :dhcp_broadcast_reply => false } end @@ -53,6 +54,7 @@ is_expected.to contain_neutron_dhcp_agent_config('DEFAULT/root_helper').with_value(p[:root_helper]); is_expected.to contain_neutron_dhcp_agent_config('DEFAULT/dhcp_delete_namespaces').with_value(p[:dhcp_delete_namespaces]); is_expected.to contain_neutron_dhcp_agent_config('DEFAULT/enable_isolated_metadata').with_value(p[:enable_isolated_metadata]); + is_expected.to contain_neutron_dhcp_agent_config('DEFAULT/force_metadata').with_value(p[:enable_force_metadata]); is_expected.to contain_neutron_dhcp_agent_config('DEFAULT/enable_metadata_network').with_value(p[:enable_metadata_network]); is_expected.to contain_neutron_dhcp_agent_config('DEFAULT/dhcp_broadcast_reply').with_value(p[:dhcp_broadcast_reply]); end @@ -112,12 +114,32 @@ end end - context 'when enabling metadata networks without enabling isolated metadata' do + context 'when enabling metadata networks without enabling isolated metadata or force metadata' do before :each do - params.merge!(:enable_isolated_metadata => false, :enable_metadata_network => true) + params.merge!(:enable_isolated_metadata => false, :enable_force_metadata => false, :enable_metadata_network => true) end - it_raises 'a Puppet::Error', /enable_metadata_network to true requires enable_isolated_metadata also enabled./ + it_raises 'a Puppet::Error', /enable_metadata_network to true requires enable_isolated_metadata or enable_force_metadata also enabled./ + end + + context 'when enabling force metadata only' do + before :each do + params.merge!(:enable_force_metadata => true, :enable_metadata_network => false) + end + it 'should enable force_metadata only' do + is_expected.to contain_neutron_dhcp_agent_config('DEFAULT/force_metadata').with_value('true'); + is_expected.to contain_neutron_dhcp_agent_config('DEFAULT/enable_metadata_network').with_value('false'); + end + end + + context 'when enabling force metadata with metadata networks' do + before :each do + params.merge!(:enable_force_metadata => true, :enable_metadata_network => true) + end + it 'should enable both force_metadata and metadata_network' do + is_expected.to contain_neutron_dhcp_agent_config('DEFAULT/force_metadata').with_value('true'); + is_expected.to contain_neutron_dhcp_agent_config('DEFAULT/enable_metadata_network').with_value('true'); + end end context 'with use_namespaces as false' do diff --git a/neutron/spec/classes/neutron_server_spec.rb b/neutron/spec/classes/neutron_server_spec.rb index ba30697b0..ba7adcb10 100644 --- a/neutron/spec/classes/neutron_server_spec.rb +++ b/neutron/spec/classes/neutron_server_spec.rb @@ -96,7 +96,7 @@ is_expected.to contain_neutron_config('DEFAULT/rpc_workers').with_value(facts[:processorcount]) is_expected.to contain_neutron_config('DEFAULT/agent_down_time').with_value(p[:agent_down_time]) is_expected.to contain_neutron_config('DEFAULT/router_scheduler_driver').with_value(p[:router_scheduler_driver]) - is_expected.to contain_neutron_config('qos/notification_drivers').with_value(['']) + is_expected.to contain_neutron_config('qos/notification_drivers').with_value(['message_queue']) end context 'with manage_service as false' do diff --git a/nova/lib/puppet/provider/nova.rb b/nova/lib/puppet/provider/nova.rb index 06b90c5a8..1faa51bb7 100644 --- a/nova/lib/puppet/provider/nova.rb +++ b/nova/lib/puppet/provider/nova.rb @@ -39,8 +39,8 @@ def nova_credentials def self.get_nova_credentials #needed keys for authentication - auth_keys = ['auth_host', 'auth_port', 'auth_protocol', - 'admin_tenant_name', 'admin_user', 'admin_password'] + auth_keys = ['auth_uri', 'admin_tenant_name', 'admin_user', + 'admin_password'] conf = nova_conf if conf and conf['keystone_authtoken'] and auth_keys.all?{|k| !conf['keystone_authtoken'][k].nil?} @@ -59,7 +59,7 @@ def self.get_nova_credentials def self.get_auth_endpoint q = nova_credentials - "#{q['auth_protocol']}://#{q['auth_host']}:#{q['auth_port']}/v2.0/" + "#{q['auth_uri']}" end def self.auth_endpoint diff --git a/nova/manifests/init.pp b/nova/manifests/init.pp index d24a6404f..a6a01674b 100644 --- a/nova/manifests/init.pp +++ b/nova/manifests/init.pp @@ -328,7 +328,10 @@ # (optional) Sets a version cap for messages sent to scheduler services # Defaults to undef # - +# [*use_ipv6*] +# (optional) Use IPv6 or not. +# Defaults to false +# class nova( $ensure_package = 'present', $database_connection = undef, @@ -405,6 +408,7 @@ $upgrade_level_intercell = undef, $upgrade_level_network = undef, $upgrade_level_scheduler = undef, + $use_ipv6 = false, ) inherits nova::params { # maintain backward compatibility @@ -681,6 +685,7 @@ 'DEFAULT/service_down_time': value => $service_down_time; 'DEFAULT/rootwrap_config': value => $rootwrap_config; 'DEFAULT/report_interval': value => $report_interval; + 'DEFAULT/use_ipv6': value => $use_ipv6; } if $notify_on_state_change and $notify_on_state_change in ['vm_state', 'vm_and_task_state'] { diff --git a/nova/spec/classes/nova_init_spec.rb b/nova/spec/classes/nova_init_spec.rb index ec8066de8..921646507 100644 --- a/nova/spec/classes/nova_init_spec.rb +++ b/nova/spec/classes/nova_init_spec.rb @@ -65,6 +65,7 @@ is_expected.to contain_nova_config('DEFAULT/notification_driver').with_ensure('absent') is_expected.to contain_nova_config('DEFAULT/rootwrap_config').with_value('/etc/nova/rootwrap.conf') is_expected.to contain_nova_config('DEFAULT/report_interval').with_value('10') + is_expected.to contain_nova_config('DEFAULT/use_ipv6').with_value('false') is_expected.to contain_nova_config('DEFAULT/os_region_name').with_ensure('absent') is_expected.to contain_nova_config('cinder/os_region_name').with_ensure('absent') is_expected.to contain_nova_config('cinder/catalog_info').with('value' => 'volumev2:cinderv2:publicURL') @@ -102,6 +103,7 @@ :notify_api_faults => true, :report_interval => '60', :os_region_name => 'MyRegion', + :use_ipv6 => true, :upgrade_level_cells => '1.0.0', :upgrade_level_cert => '1.0.0', :upgrade_level_compute => '1.0.0', @@ -163,6 +165,7 @@ is_expected.to contain_nova_config('DEFAULT/notification_topics').with_value('openstack') is_expected.to contain_nova_config('DEFAULT/notify_api_faults').with_value(true) is_expected.to contain_nova_config('DEFAULT/report_interval').with_value('60') + is_expected.to contain_nova_config('DEFAULT/use_ipv6').with_value('true') is_expected.to contain_nova_config('cinder/os_region_name').with_value('MyRegion') is_expected.to contain_nova_config('DEFAULT/os_region_name').with_ensure('absent') end diff --git a/nova/spec/unit/provider/nova_spec.rb b/nova/spec/unit/provider/nova_spec.rb index 46608a0df..acd3bd9ac 100644 --- a/nova/spec/unit/provider/nova_spec.rb +++ b/nova/spec/unit/provider/nova_spec.rb @@ -11,9 +11,7 @@ def klass let :credential_hash do { - 'auth_host' => '192.168.56.210', - 'auth_port' => '35357', - 'auth_protocol' => 'https', + 'auth_uri' => 'https://192.168.56.210:35357/v2.0/', 'admin_tenant_name' => 'admin_tenant', 'admin_user' => 'admin', 'admin_password' => 'password', @@ -58,7 +56,7 @@ def klass end.to raise_error(Puppet::Error, credential_error) end - it 'should use specified host/port/protocol in the auth endpoint' do + it 'should use specified uri in the auth endpoint' do conf = {'keystone_authtoken' => credential_hash} klass.expects(:nova_conf).returns(conf) expect(klass.get_auth_endpoint).to eq(auth_endpoint) diff --git a/trove/manifests/db.pp b/trove/manifests/db.pp index 17a1faffc..9368ba7e4 100644 --- a/trove/manifests/db.pp +++ b/trove/manifests/db.pp @@ -43,6 +43,8 @@ $database_max_overflow = 20, ) { + include ::trove::params + # NOTE(spredzy): In order to keep backward compatibility we rely on the pick function # to use trove:: if trove::db:: isn't specified. $database_connection_real = pick($::trove::database_connection, $database_connection) @@ -54,14 +56,18 @@ $database_max_overflow_real = pick($::trove::database_max_overflow, $database_max_overflow) validate_re($database_connection_real, - '(sqlite|mysql|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') + '^(sqlite|mysql(\+pymysql)?|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') if $database_connection_real { case $database_connection_real { - /^mysql:\/\//: { - $backend_package = false + /^mysql(\+pymysql)?:\/\//: { require 'mysql::bindings' require 'mysql::bindings::python' + if $database_connection_real =~ /^mysql\+pymysql/ { + $backend_package = $::trove::params::pymysql_package_name + } else { + $backend_package = false + } } /^postgresql:\/\//: { $backend_package = false diff --git a/trove/manifests/params.pp b/trove/manifests/params.pp index c1192e7a4..809f5d2f7 100644 --- a/trove/manifests/params.pp +++ b/trove/manifests/params.pp @@ -15,6 +15,7 @@ $taskmanager_package_name = 'openstack-trove-taskmanager' $taskmanager_service_name = 'openstack-trove-taskmanager' $sqlite_package_name = undef + $pymysql_package_name = undef } 'Debian': { $client_package_name = 'python-troveclient' @@ -28,6 +29,7 @@ $taskmanager_package_name = 'trove-taskmanager' $taskmanager_service_name = 'trove-taskmanager' $sqlite_package_name = 'python-pysqlite2' + $pymysql_package_name = 'python-pymysql' } default: { fail("Unsupported osfamily: ${::osfamily} operatingsystem") diff --git a/trove/spec/acceptance/basic_trove_spec.rb b/trove/spec/acceptance/basic_trove_spec.rb index 0a30141d1..c51fcecda 100644 --- a/trove/spec/acceptance/basic_trove_spec.rb +++ b/trove/spec/acceptance/basic_trove_spec.rb @@ -29,7 +29,7 @@ # Trove resources class { '::trove': - database_connection => 'mysql://trove:a_big_secret@127.0.0.1/trove?charset=utf8', + database_connection => 'mysql+pymysql://trove:a_big_secret@127.0.0.1/trove?charset=utf8', rabbit_userid => 'trove', rabbit_password => 'an_even_bigger_secret', rabbit_host => '127.0.0.1', diff --git a/trove/spec/classes/trove_db_spec.rb b/trove/spec/classes/trove_db_spec.rb index f297a8311..f8d277ca2 100644 --- a/trove/spec/classes/trove_db_spec.rb +++ b/trove/spec/classes/trove_db_spec.rb @@ -18,7 +18,7 @@ context 'with specific parameters' do let :params do - { :database_connection => 'mysql://trove:trove@localhost/trove', + { :database_connection => 'mysql+pymysql://trove:trove@localhost/trove', :database_idle_timeout => '3601', :database_min_pool_size => '2', :database_max_pool_size => '21', @@ -27,14 +27,21 @@ :database_retry_interval => '11', } end - it { is_expected.to contain_trove_config('database/connection').with_value('mysql://trove:trove@localhost/trove').with_secret(true) } + it { is_expected.to contain_trove_config('database/connection').with_value('mysql+pymysql://trove:trove@localhost/trove').with_secret(true) } it { is_expected.to contain_trove_config('database/idle_timeout').with_value('3601') } it { is_expected.to contain_trove_config('database/min_pool_size').with_value('2') } it { is_expected.to contain_trove_config('database/max_retries').with_value('11') } it { is_expected.to contain_trove_config('database/max_pool_size').with_value('21') } it { is_expected.to contain_trove_config('database/max_overflow').with_value('21') } it { is_expected.to contain_trove_config('database/retry_interval').with_value('11') } + end + + context 'with MySQL-python library as backend package' do + let :params do + { :database_connection => 'mysql://trove:trove@localhost/trove', } + end + it { is_expected.to contain_package('python-mysqldb').with(:ensure => 'present') } end context 'with postgresql backend' do @@ -56,6 +63,14 @@ it_raises 'a Puppet::Error', /validate_re/ end + context 'with incorrect pymysql database_connection string' do + let :params do + { :database_connection => 'foo+pymysql://trove:trove@localhost/trove', } + end + + it_raises 'a Puppet::Error', /validate_re/ + end + end context 'on Debian platforms' do @@ -67,6 +82,20 @@ end it_configures 'trove::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://trove:trove@localhost/trove', } + end + + it 'install the proper backend package' do + is_expected.to contain_package('trove-backend-package').with( + :ensure => 'present', + :name => 'python-pymysql', + :tag => 'openstack' + ) + end + end end context 'on Redhat platforms' do @@ -77,6 +106,14 @@ end it_configures 'trove::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://trove:trove@localhost/trove', } + end + + it { is_expected.not_to contain_package('trove-backend-package') } + end end end diff --git a/vswitch/lib/puppet/provider/vs_port/ovs_redhat.rb b/vswitch/lib/puppet/provider/vs_port/ovs_redhat.rb index c4a3205c1..0134bfa38 100644 --- a/vswitch/lib/puppet/provider/vs_port/ovs_redhat.rb +++ b/vswitch/lib/puppet/provider/vs_port/ovs_redhat.rb @@ -1,16 +1,6 @@ require File.expand_path(File.join(File.dirname(__FILE__), '..', '..', '..', 'puppetx', 'redhat', 'ifcfg.rb')) require File.expand_path(File.join(File.dirname(__FILE__), '.','ovs.rb')) -BASE = '/etc/sysconfig/network-scripts/ifcfg-' - -# When not seedling from interface file -DEFAULT = { - 'ONBOOT' => 'yes', - 'BOOTPROTO' => 'dhcp', - 'PEERDNS' => 'no', - 'NM_CONTROLLED' => 'no', - 'NOZEROCONF' => 'yes' } - Puppet::Type.type(:vs_port).provide( :ovs_redhat, :parent => Puppet::Type.type(:vs_port).provider(:ovs) @@ -18,6 +8,17 @@ desc 'Openvswitch port manipulation for RedHat OSes family' + self::BASE = '/etc/sysconfig/network-scripts/ifcfg-' + + # When not seedling from interface file + self::DEFAULT = { + 'ONBOOT' => 'yes', + 'BOOTPROTO' => 'dhcp', + 'PEERDNS' => 'no', + 'NM_CONTROLLED' => 'no', + 'NOZEROCONF' => 'yes' + } + confine :osfamily => :redhat defaultfor :osfamily => :redhat From aaa1a0995580c26863c050bb45a477c7b36f3caf Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Thu, 21 Jan 2016 15:34:18 -0500 Subject: [PATCH 16/35] Update mongodb to b25bdb3ce7dcb3e60a4427806fe72cc93625de13 b25bdb3ce7dcb3e60a4427806fe72cc93625de13 Merge pull request #258 from EmilienM/mongo/provider 2a89243a75ee3be1665f1ab545900afbd882a941 Fix mongodb provider 26c2928f5fae8ed2414356668929db09dacabb68 Merge pull request #257 from EmilienM/mongo/fixshell e372d3e84c57ff4f256c6303f32da5c60587b668 mongodb: do not add blank parameter in ipv4 202ee76e281ba8f27bbcfd5f39848d2242619cb1 Merge pull request #255 from EmilienM/providers/ipv6 dc31d79e1a39749577ee0385b3bcddc7e7248c41 Merge pull request #256 from hunner/fix_versioncmp a04f37d689d8425336dd84a54bed34a90d57c829 Fix versioncmp when version is undef a77d1a191419a3f1f468707aefa69abd689c3e4b (MODULES-2983) Enable IPv6 in mongodb provider 3bcfc75229c4faffe5ccfe9caf1278a54ef0f7cc Merge pull request #237 from erikanderson/normalize_template_spacing 38ceb81e36bbbb844039e12fb8aebb43cb2cef0d Merge pull request #232 from GoozeyX/squashed_forpull 2537fc4dc770c1e1a46567691aa7e7f76961fb52 added yum proxy options 9b9f7571aed4401197437659b31d8d5ac3849e4e Normalize spacing in template Change-Id: Ib733a303d40c9a32f8247f47c2ddc41c60796cac --- Puppetfile | 2 +- mongodb/CHANGELOG.md | 4 ++ mongodb/README.md | 9 ++++ mongodb/lib/puppet/provider/mongodb.rb | 36 ++++++++++++++-- mongodb/manifests/globals.pp | 4 ++ mongodb/manifests/params.pp | 48 +++++++++++---------- mongodb/manifests/repo.pp | 11 +++-- mongodb/manifests/repo/yum.pp | 11 +++-- mongodb/manifests/server/config.pp | 2 +- mongodb/spec/classes/mongos_service_spec.rb | 20 +++++---- mongodb/spec/classes/repo_spec.rb | 39 ++++++++++++++--- mongodb/templates/mongodb.conf.2.6.erb | 8 ++-- 12 files changed, 140 insertions(+), 54 deletions(-) diff --git a/Puppetfile b/Puppetfile index 895825747..bb2d1d29f 100644 --- a/Puppetfile +++ b/Puppetfile @@ -147,7 +147,7 @@ mod 'module-data', :git => 'https://github.com/ripienaar/puppet-module-data.git' mod 'mongodb', - :commit => 'a5d6e5d36fb1007534bca85fd277a678e6c5a2ee', + :commit => 'b25bdb3ce7dcb3e60a4427806fe72cc93625de13', :git => 'https://github.com/puppetlabs/puppetlabs-mongodb.git' mod 'mysql', diff --git a/mongodb/CHANGELOG.md b/mongodb/CHANGELOG.md index 57530a055..218d0e911 100644 --- a/mongodb/CHANGELOG.md +++ b/mongodb/CHANGELOG.md @@ -1,3 +1,7 @@ +## Unreleased +### Summary +- support setting a proxy for yum repositories with or without user/password authentication + ## 2015-06-22 - Release 0.11.0 ### Summary diff --git a/mongodb/README.md b/mongodb/README.md index 44fc3ee4a..5753fa888 100644 --- a/mongodb/README.md +++ b/mongodb/README.md @@ -232,6 +232,15 @@ the module will use the default for your OS distro. This setting can be used to override the default MongoDB repository location. If not specified, the module will use the default repository for your OS distro. +#####`repo_proxy` +This will allow you to set a proxy for your repository in case you are behind a corporate firewall. Currently this is only supported with yum repositories + +#####`proxy_username` +This sets the username for the proxyserver, should authentication be required + +#####`proxy_password` +This sets the password for the proxyserver, should authentication be required + ####Class: mongodb::server Most of the parameters manipulate the mongod.conf file. diff --git a/mongodb/lib/puppet/provider/mongodb.rb b/mongodb/lib/puppet/provider/mongodb.rb index 6bbc350b6..f501d6217 100644 --- a/mongodb/lib/puppet/provider/mongodb.rb +++ b/mongodb/lib/puppet/provider/mongodb.rb @@ -28,6 +28,30 @@ def self.get_mongod_conf_file file end + def self.ipv6_is_enabled + file = get_mongod_conf_file + config = YAML.load_file(file) + if config.kind_of?(Hash) + ipv6 = config['net.ipv6'] + else # It has to be a key-value store + config = {} + File.readlines(file).collect do |line| + k,v = line.split('=') + config[k.rstrip] = v.lstrip.chomp if k and v + end + ipv6 = config['ipv6'] + end + ipv6 + end + + def self.mongo_cmd(db, host, cmd) + if ipv6_is_enabled + out = mongo([db, '--quiet', '--ipv6', '--host', host, '--eval', cmd]) + else + out = mongo([db, '--quiet', '--host', host, '--eval', cmd]) + end + end + def self.get_conn_string file = get_mongod_conf_file # The mongo conf is probably a key-value store, even though 2.6 is @@ -55,8 +79,11 @@ def self.get_conn_string if bindip first_ip_in_list = bindip.split(',').first - if first_ip_in_list.eql? '0.0.0.0' + case first_ip_in_list + when '0.0.0.0' ip_real = '127.0.0.1' + when /\[?::0\]?/ + ip_real = '::1' else ip_real = first_ip_in_list end @@ -80,7 +107,8 @@ def self.db_ismaster if mongorc_file cmd_ismaster = mongorc_file + cmd_ismaster end - out = mongo(['admin', '--quiet', '--host', get_conn_string, '--eval', cmd_ismaster]) + db = 'admin' + out = mongo_cmd(db, get_conn_string, cmd_ismaster) out.gsub!(/ObjectId\(([^)]*)\)/, '\1') out.gsub!(/ISODate\((.+?)\)/, '\1 ') out.gsub!(/^Error\:.+/, '') @@ -122,9 +150,9 @@ def self.mongo_eval(cmd, db = 'admin', retries = 10, host = nil) retry_count.times do |n| begin if host - out = mongo([db, '--quiet', '--host', host, '--eval', cmd]) + out = mongo_cmd(db, host, cmd) else - out = mongo([db, '--quiet', '--host', get_conn_string, '--eval', cmd]) + out = mongo_cmd(db, get_conn_string, cmd) end rescue => e Puppet.debug "Request failed: '#{e.message}' Retry: '#{n}'" diff --git a/mongodb/manifests/globals.pp b/mongodb/manifests/globals.pp index 5bb33b9c5..85f418b0b 100644 --- a/mongodb/manifests/globals.pp +++ b/mongodb/manifests/globals.pp @@ -27,6 +27,9 @@ $manage_package_repo = undef, $manage_package = undef, + $repo_proxy = undef, + $proxy_username = undef, + $proxy_password = undef, $repo_location = undef, $use_enterprise_repo = undef, @@ -39,6 +42,7 @@ class { '::mongodb::repo': ensure => present, repo_location => $repo_location, + proxy => $repo_proxy, } } } diff --git a/mongodb/manifests/params.pp b/mongodb/manifests/params.pp index 0926f3946..7f47765b4 100644 --- a/mongodb/manifests/params.pp +++ b/mongodb/manifests/params.pp @@ -22,6 +22,8 @@ $manage_package = pick($mongodb::globals::manage_package, $mongodb::globals::manage_package_repo, false) + $version = $::mongodb::globals::version + # Amazon Linux's OS Family is 'Linux', operating system 'Amazon'. case $::osfamily { 'RedHat', 'Linux': { @@ -29,7 +31,7 @@ if $manage_package { $user = pick($::mongodb::globals::user, 'mongod') $group = pick($::mongodb::globals::group, 'mongod') - if ($::mongodb::globals::version == undef) { + if ($version == undef) { $server_package_name = pick($::mongodb::globals::server_package_name, 'mongodb-org-server') $client_package_name = pick($::mongodb::globals::client_package_name, 'mongodb-org-shell') $mongos_package_name = pick($::mongodb::globals::mongos_package_name, 'mongodb-org-mongos') @@ -38,20 +40,20 @@ $package_ensure_mongos = true } else { # check if the version is greater than 2.6 - if(versioncmp($::mongodb::globals::version, '2.6.0') >= 0) { + if $version and (versioncmp($version, '2.6.0') >= 0) { $server_package_name = pick($::mongodb::globals::server_package_name, 'mongodb-org-server') $client_package_name = pick($::mongodb::globals::client_package_name, 'mongodb-org-shell') $mongos_package_name = pick($::mongodb::globals::mongos_package_name, 'mongodb-org-mongos') - $package_ensure = $::mongodb::globals::version - $package_ensure_client = $::mongodb::globals::version - $package_ensure_mongos = $::mongodb::globals::version + $package_ensure = $version + $package_ensure_client = $version + $package_ensure_mongos = $version } else { $server_package_name = pick($::mongodb::globals::server_package_name, 'mongodb-10gen') $client_package_name = pick($::mongodb::globals::client_package_name, 'mongodb-10gen') $mongos_package_name = pick($::mongodb::globals::mongos_package_name, 'mongodb-10gen') - $package_ensure = $::mongodb::globals::version - $package_ensure_client = $::mongodb::globals::version #this is still needed in case they are only installing the client - $package_ensure_mongos = $::mongodb::globals::version + $package_ensure = $version + $package_ensure_client = $version #this is still needed in case they are only installing the client + $package_ensure_mongos = $version } } $service_name = pick($::mongodb::globals::service_name, 'mongod') @@ -70,14 +72,14 @@ } else { # RedHat/CentOS doesn't come with a prepacked mongodb # so we assume that you are using EPEL repository. - if ($::mongodb::globals::version == undef) { + if ($version == undef) { $package_ensure = true $package_ensure_client = true $package_ensure_mongos = true } else { - $package_ensure = $::mongodb::globals::version - $package_ensure_client = $::mongodb::globals::version - $package_ensure_mongos = $::mongodb::globals::version + $package_ensure = $version + $package_ensure_client = $version + $package_ensure_mongos = $version } $user = pick($::mongodb::globals::user, 'mongodb') $group = pick($::mongodb::globals::group, 'mongodb') @@ -114,7 +116,7 @@ if $manage_package { $user = pick($::mongodb::globals::user, 'mongodb') $group = pick($::mongodb::globals::group, 'mongodb') - if ($::mongodb::globals::version == undef) { + if ($version == undef) { $server_package_name = pick($::mongodb::globals::server_package_name, 'mongodb-org-server') $client_package_name = pick($::mongodb::globals::client_package_name, 'mongodb-org-shell') $mongos_package_name = pick($::mongodb::globals::mongos_package_name, 'mongodb-org-mongos') @@ -125,21 +127,21 @@ $config = '/etc/mongod.conf' } else { # check if the version is greater than 2.6 - if(versioncmp($::mongodb::globals::version, '2.6.0') >= 0) { + if $version and (versioncmp($version, '2.6.0') >= 0) { $server_package_name = pick($::mongodb::globals::server_package_name, 'mongodb-org-server') $client_package_name = pick($::mongodb::globals::client_package_name, 'mongodb-org-shell') $mongos_package_name = pick($::mongodb::globals::mongos_package_name, 'mongodb-org-mongos') - $package_ensure = $::mongodb::globals::version - $package_ensure_client = $::mongodb::globals::version - $package_ensure_mongos = $::mongodb::globals::version + $package_ensure = $version + $package_ensure_client = $version + $package_ensure_mongos = $version $service_name = pick($::mongodb::globals::service_name, 'mongod') $config = '/etc/mongod.conf' } else { $server_package_name = pick($::mongodb::globals::server_package_name, 'mongodb-10gen') $client_package_name = pick($::mongodb::globals::client_package_name, 'mongodb-10gen') $mongos_package_name = pick($::mongodb::globals::mongos_package_name, 'mongodb-10gen') - $package_ensure = $::mongodb::globals::version - $package_ensure_client = $::mongodb::globals::version #this is still needed in case they are only installing the client + $package_ensure = $version + $package_ensure_client = $version #this is still needed in case they are only installing the client $service_name = pick($::mongodb::globals::service_name, 'mongodb') $config = '/etc/mongodb.conf' } @@ -155,14 +157,14 @@ # I would not recommend to use the prepacked # mongodb server on Ubuntu 12.04 or Debian 6/7, # because its really outdated - if ($::mongodb::globals::version == undef) { + if ($version == undef) { $package_ensure = true $package_ensure_client = true $package_ensure_mongos = true } else { - $package_ensure = $::mongodb::globals::version - $package_ensure_client = $::mongodb::globals::version - $package_ensure_mongos = $::mongodb::globals::version + $package_ensure = $version + $package_ensure_client = $version + $package_ensure_mongos = $version } $user = pick($::mongodb::globals::user, 'mongodb') $group = pick($::mongodb::globals::group, 'mongodb') diff --git a/mongodb/manifests/repo.pp b/mongodb/manifests/repo.pp index e117ecc5e..aa9a99e33 100644 --- a/mongodb/manifests/repo.pp +++ b/mongodb/manifests/repo.pp @@ -1,8 +1,11 @@ # PRIVATE CLASS: do not use directly class mongodb::repo ( - $ensure = $mongodb::params::ensure, - $version = $mongodb::params::version, - $repo_location = undef, + $ensure = $mongodb::params::ensure, + $version = $mongodb::params::version, + $repo_location = undef, + $proxy = undef, + $proxy_username = undef, + $proxy_password = undef, ) inherits mongodb::params { case $::osfamily { 'RedHat', 'Linux': { @@ -13,7 +16,7 @@ $location = 'https://repo.mongodb.com/yum/redhat/$releasever/mongodb-enterprise/stable/$basearch/' $description = 'MongoDB Enterprise Repository' } - elsif (versioncmp($version, '3.0.0') >= 0) { + elsif $version and (versioncmp($version, '3.0.0') >= 0) { $mongover = split($version, '[.]') $location = $::architecture ? { 'x86_64' => "http://repo.mongodb.org/yum/redhat/${::operatingsystemmajrelease}/mongodb-org/${mongover[0]}.${mongover[1]}/x86_64/", diff --git a/mongodb/manifests/repo/yum.pp b/mongodb/manifests/repo/yum.pp index cc033327f..6046e9a3e 100644 --- a/mongodb/manifests/repo/yum.pp +++ b/mongodb/manifests/repo/yum.pp @@ -5,10 +5,13 @@ if($::mongodb::repo::ensure == 'present' or $::mongodb::repo::ensure == true) { yumrepo { 'mongodb': - descr => $::mongodb::repo::description, - baseurl => $::mongodb::repo::location, - gpgcheck => '0', - enabled => '1', + descr => $::mongodb::repo::description, + baseurl => $::mongodb::repo::location, + gpgcheck => '0', + enabled => '1', + proxy => $::mongodb::repo::proxy, + proxy_username => $::mongodb::repo::proxy_username, + proxy_password => $::mongodb::repo::proxy_password, } Yumrepo['mongodb'] -> Package<|tag == 'mongodb'|> } diff --git a/mongodb/manifests/server/config.pp b/mongodb/manifests/server/config.pp index 6c9df389e..636a12e44 100644 --- a/mongodb/manifests/server/config.pp +++ b/mongodb/manifests/server/config.pp @@ -100,7 +100,7 @@ #Pick which config content to use if $config_content { $cfg_content = $config_content - } elsif (versioncmp($version, '2.6.0') >= 0) { + } elsif $version and (versioncmp($version, '2.6.0') >= 0) { # Template uses: # - $auth # - $bind_ip diff --git a/mongodb/spec/classes/mongos_service_spec.rb b/mongodb/spec/classes/mongos_service_spec.rb index b95582944..41730d0c8 100644 --- a/mongodb/spec/classes/mongos_service_spec.rb +++ b/mongodb/spec/classes/mongos_service_spec.rb @@ -5,8 +5,9 @@ context 'on Debian with service_manage set to true' do let :facts do { - :osfamily => 'Debian', - :operatingsystem => 'Debian', + :osfamily => 'Debian', + :operatingsystem => 'Debian', + :operatingsystemrelease => '7.0', } end @@ -29,8 +30,9 @@ context 'on Debian with service_manage set to false' do let :facts do { - :osfamily => 'Debian', - :operatingsystem => 'Debian', + :osfamily => 'Debian', + :operatingsystem => 'Debian', + :operatingsystemrelease => '7.0', } end @@ -50,8 +52,9 @@ context 'on RedHat with service_manage set to true' do let :facts do { - :osfamily => 'RedHat', - :operatingsystem => 'RedHat', + :osfamily => 'RedHat', + :operatingsystem => 'RedHat', + :operatingsystemrelease => '7.0', } end @@ -78,8 +81,9 @@ context 'on RedHat with service_manage set to false' do let :facts do { - :osfamily => 'RedHat', - :operatingsystem => 'RedHat', + :osfamily => 'RedHat', + :operatingsystem => 'RedHat', + :operatingsystemrelease => '7.0', } end diff --git a/mongodb/spec/classes/repo_spec.rb b/mongodb/spec/classes/repo_spec.rb index ef8b11377..79e14a5cd 100644 --- a/mongodb/spec/classes/repo_spec.rb +++ b/mongodb/spec/classes/repo_spec.rb @@ -5,9 +5,10 @@ context 'when deploying on Debian' do let :facts do { - :osfamily => 'Debian', - :operatingsystem => 'Debian', - :lsbdistid => 'Debian', + :osfamily => 'Debian', + :operatingsystem => 'Debian', + :operatingsystemrelease => '7.0', + :lsbdistid => 'Debian', } end @@ -19,8 +20,9 @@ context 'when deploying on CentOS' do let :facts do { - :osfamily => 'RedHat', - :operatingsystem => 'CentOS', + :osfamily => 'RedHat', + :operatingsystem => 'CentOS', + :operatingsystemrelease => '7.0', } end @@ -29,4 +31,31 @@ } end + context 'when yumrepo has a proxy set' do + let :facts do + { + :osfamily => 'RedHat', + :operatingsystem => 'RedHat', + :operatingsystemrelease => '7.0', + } + end + let :params do + { + :proxy => 'http://proxy-server:8080', + :proxy_username => 'proxyuser1', + :proxy_password => 'proxypassword1', + } + end + it { + is_expected.to contain_class('mongodb::repo::yum') + } + it do + should contain_yumrepo('mongodb').with({ + 'enabled' => '1', + 'proxy' => 'http://proxy-server:8080', + 'proxy_username' => 'proxyuser1', + 'proxy_password' => 'proxypassword1', + }) + end + end end diff --git a/mongodb/templates/mongodb.conf.2.6.erb b/mongodb/templates/mongodb.conf.2.6.erb index 293f8099a..0f1775562 100644 --- a/mongodb/templates/mongodb.conf.2.6.erb +++ b/mongodb/templates/mongodb.conf.2.6.erb @@ -1,6 +1,6 @@ -# mongo.conf - generated from Puppet +#mongo.conf - generated from Puppet -# System Log +#System Log <% if @logpath -%> systemLog.path: <%= @logpath %> @@ -80,7 +80,7 @@ security.javascriptEnabled: <%= @noscripting %> <% end -%> -# Net +#Net <% if @ipv6 -%> net.ipv6=<%= @ipv6 %> <% end -%> @@ -111,7 +111,7 @@ replication.replSetName: <%= @replset %> replication.oplogSizeMB: <%= @oplog_size %> <% end -%> -# Sharding +#Sharding <% if @configsvr -%> sharding.clusterRole: configsvr <% end -%> From a1f7096f30aa5397d93f2f5c34cc31e909281fe0 Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Thu, 21 Jan 2016 15:35:43 -0500 Subject: [PATCH 17/35] Sync liberty openstack puppet modules. Update keystone to eed17a8460994382c14c2573e311c528fe3bd761 eed17a8460994382c14c2573e311c528fe3bd761 Fix region association for endpoints. Change-Id: Ifc40bc8a3cb03f4c66b540b5bf33395e1b3a4a03 Update tripleo to e40d5282833ddfa2c700044440b7c96101736453 e40d5282833ddfa2c700044440b7c96101736453 Enable X-Forwarded-Proto header for cinder Change-Id: Idc354674a2546669f9b0cb9279bdadea3aee1adf Update vswitch to 906abc6dd77218e8734a9e7c6101beb951d4d627 906abc6dd77218e8734a9e7c6101beb951d4d627 Try to use zuul-cloner to prepare fixtures Change-Id: I8d2c51258639507c1ba1a65e677a1540fc628ed8 --- Puppetfile | 6 +- .../provider/keystone_endpoint/openstack.rb | 3 +- .../spec/classes/keystone_endpoint_spec.rb | 12 ++- ...keystone_resource_service_identity_spec.rb | 17 ++++ .../matchers/have_array_of_instances_hash.rb | 17 ++++ .../keystone_endpoint/openstack_spec.rb | 94 +++++++++++++++++-- tripleo/manifests/loadbalancer.pp | 4 + vswitch/.fixtures.yml | 3 - vswitch/.gitignore | 1 + vswitch/Gemfile | 1 + vswitch/Rakefile | 64 +++++++++++++ 11 files changed, 205 insertions(+), 17 deletions(-) create mode 100644 keystone/spec/support/matchers/have_array_of_instances_hash.rb delete mode 100644 vswitch/.fixtures.yml diff --git a/Puppetfile b/Puppetfile index bb2d1d29f..d7f22e981 100644 --- a/Puppetfile +++ b/Puppetfile @@ -119,7 +119,7 @@ mod 'keepalived', :git => 'https://github.com/Unyonsys/puppet-module-keepalived.git' mod 'keystone', - :commit => 'def536208d70b70171a8f6b5797077b0638abde9', + :commit => 'eed17a8460994382c14c2573e311c528fe3bd761', :git => 'https://github.com/openstack/puppet-keystone.git' mod 'kibana3', @@ -263,7 +263,7 @@ mod 'tomcat', :git => 'https://github.com/puppetlabs/puppetlabs-tomcat.git' mod 'tripleo', - :commit => '8ebfc67734b6525f9a03aad9617d367d68780dac', + :commit => 'e40d5282833ddfa2c700044440b7c96101736453', :git => 'https://github.com/openstack/puppet-tripleo.git' mod 'trove', @@ -287,7 +287,7 @@ mod 'vlan', :git => 'https://github.com/derekhiggins/puppet-vlan.git' mod 'vswitch', - :commit => '0abd9227ed509e5aa63aa52eba991e6bee13f67e', + :commit => '906abc6dd77218e8734a9e7c6101beb951d4d627', :git => 'https://github.com/openstack/puppet-vswitch.git' mod 'xinetd', diff --git a/keystone/lib/puppet/provider/keystone_endpoint/openstack.rb b/keystone/lib/puppet/provider/keystone_endpoint/openstack.rb index 410d49291..93d6fb419 100644 --- a/keystone/lib/puppet/provider/keystone_endpoint/openstack.rb +++ b/keystone/lib/puppet/provider/keystone_endpoint/openstack.rb @@ -98,7 +98,8 @@ def self.instances endpoints.each do |ep_osc| if (ep_osc[:id] != current[:id]) && (ep_osc[:service_name] == current[:service_name]) && - (ep_osc[:service_type] == current[:service_type]) + (ep_osc[:service_type] == current[:service_type]) && + (ep_osc[:region] == current[:region]) endpoint.merge!(ep_osc[:interface].to_sym => ep_osc) end end diff --git a/keystone/spec/classes/keystone_endpoint_spec.rb b/keystone/spec/classes/keystone_endpoint_spec.rb index ec6f6751d..efc71eb4f 100644 --- a/keystone/spec/classes/keystone_endpoint_spec.rb +++ b/keystone/spec/classes/keystone_endpoint_spec.rb @@ -12,7 +12,8 @@ :ensure => 'present', :public_url => 'http://127.0.0.1:5000/v2.0', :admin_url => 'http://127.0.0.1:35357/v2.0', - :internal_url => 'http://127.0.0.1:5000/v2.0' + :internal_url => 'http://127.0.0.1:5000/v2.0', + :region => 'RegionOne' )} end @@ -22,14 +23,17 @@ { :version => 'v42.6', :public_url => 'https://identity.some.tld/the/main/endpoint', :admin_url => 'https://identity-int.some.tld/some/admin/endpoint', - :internal_url => 'https://identity-int.some.tld/some/internal/endpoint' } + :internal_url => 'https://identity-int.some.tld/some/internal/endpoint', + :region => 'East' + } end - it { is_expected.to contain_keystone_endpoint('RegionOne/keystone::identity').with( + it { is_expected.to contain_keystone_endpoint('East/keystone::identity').with( :ensure => 'present', :public_url => 'https://identity.some.tld/the/main/endpoint/v42.6', :admin_url => 'https://identity-int.some.tld/some/admin/endpoint/v42.6', - :internal_url => 'https://identity-int.some.tld/some/internal/endpoint/v42.6' + :internal_url => 'https://identity-int.some.tld/some/internal/endpoint/v42.6', + :region => 'East' )} end diff --git a/keystone/spec/defines/keystone_resource_service_identity_spec.rb b/keystone/spec/defines/keystone_resource_service_identity_spec.rb index 2e5834b4d..0677d2aac 100644 --- a/keystone/spec/defines/keystone_resource_service_identity_spec.rb +++ b/keystone/spec/defines/keystone_resource_service_identity_spec.rb @@ -57,6 +57,22 @@ :public_url => 'http://7.7.7.7:9696', :internal_url => 'http://10.0.0.1:9696', :admin_url => 'http://192.168.0.1:9696', + :region => 'RegionOne', + )} + end + + context 'when explicitly setting an region' do + let :params do + required_params.merge( + :region => 'East', + ) + end + it { is_expected.to contain_keystone_endpoint("East/#{title}::network").with( + :ensure => 'present', + :public_url => 'http://7.7.7.7:9696', + :internal_url => 'http://10.0.0.1:9696', + :admin_url => 'http://192.168.0.1:9696', + :region => 'East', )} end @@ -109,6 +125,7 @@ :roles => ['admin'], )} end + context 'with user and project domain' do let :params do required_params.merge({ diff --git a/keystone/spec/support/matchers/have_array_of_instances_hash.rb b/keystone/spec/support/matchers/have_array_of_instances_hash.rb new file mode 100644 index 000000000..75273321d --- /dev/null +++ b/keystone/spec/support/matchers/have_array_of_instances_hash.rb @@ -0,0 +1,17 @@ +RSpec::Matchers.define :have_array_of_instances_hash do |expected| + expected_array = expected.dup + match do |actual| + if actual.count != expected_array.count + return false + end + actual_array = actual.map { |i| i.instance_variable_get('@property_hash') } + expected_array.each do |e| + actual_array.each do |a| + if e == a + actual_array.delete(a) + end + end + end + actual_array.empty? + end +end diff --git a/keystone/spec/unit/provider/keystone_endpoint/openstack_spec.rb b/keystone/spec/unit/provider/keystone_endpoint/openstack_spec.rb index 9bc38922d..36cb7110a 100644 --- a/keystone/spec/unit/provider/keystone_endpoint/openstack_spec.rb +++ b/keystone/spec/unit/provider/keystone_endpoint/openstack_spec.rb @@ -118,16 +118,98 @@ end describe '#instances' do - it 'finds every tenant' do - described_class.expects(:openstack) - .with('endpoint', 'list', '--quiet', '--format', 'csv', []) - .returns('"ID","Region","Service Name","Service Type","Enabled","Interface","URL" + context 'basic' do + it 'finds every tenant' do + described_class.expects(:openstack) + .with('endpoint', 'list', '--quiet', '--format', 'csv', []) + .returns('"ID","Region","Service Name","Service Type","Enabled","Interface","URL" "endpoint1_id","RegionOne","keystone","identity",True,"admin","http://127.0.0.1:5002" "endpoint2_id","RegionOne","keystone","identity",True,"internal","https://127.0.0.1:5001" "endpoint3_id","RegionOne","keystone","identity",True,"public","https://127.0.0.1:5000" ') - instances = described_class.instances - expect(instances.count).to eq(1) + instances = described_class.instances + expect(instances.count).to eq(1) + end + end + context 'many different region' do + it 'should not mix up the endpoints' do + described_class.expects(:openstack) + .with('endpoint', 'list', '--quiet', '--format', 'csv', []) + .returns('"ID","Region","Service Name","Service Type","Enabled","Interface","URL" +"endpoint1_id","RegionOne","keystone","identity",True,"admin","http://One-127.0.0.1:5002" +"endpoint2_id","RegionOne","keystone","identity",True,"internal","https://One-127.0.0.1:5001" +"endpoint3_id","RegionOne","keystone","identity",True,"public","https://One-127.0.0.1:5000" +"endpoint4_id","RegionTwo","keystone","identity",True,"admin","http://Two-127.0.0.1:5002" +"endpoint5_id","RegionTwo","keystone","identity",True,"internal","https://Two-127.0.0.1:5001" +"endpoint6_id","RegionTwo","keystone","identity",True,"public","https://Two-127.0.0.1:5000" +"endpoint7_id","RegionThree","keystone","identity",True,"admin","http://Three-127.0.0.1:5002" +"endpoint8_id","RegionThree","keystone","identity",True,"internal","https://Three-127.0.0.1:5001" +"endpoint9_id","RegionThree","keystone","identity",True,"public","https://Three-127.0.0.1:5000" +"endpoint10_id","RegionFour","keystone","identity",True,"admin","http://Four-127.0.0.1:5002" +"endpoint11_id","RegionFour","keystone","identity",True,"internal","https://Four-127.0.0.1:5001" +"endpoint12_id","RegionFour","keystone","identity",True,"public","https://Four-127.0.0.1:5000" +"endpoint13_id","RegionFive","keystone","identity",True,"admin","http://Five-127.0.0.1:5002" +"endpoint14_id","RegionFive","keystone","identity",True,"internal","https://Five-127.0.0.1:5001" +"endpoint15_id","RegionFive","keystone","identity",True,"public","https://Five-127.0.0.1:5000" +"endpoint16_id","RegionSix","keystone","identity",True,"admin","http://Six-127.0.0.1:5002" +"endpoint17_id","RegionSix","keystone","identity",True,"internal","https://Six-127.0.0.1:5001" +"endpoint18_id","RegionSix","keystone","identity",True,"public","https://Six-127.0.0.1:5000" +"endpoint19_id","RegionSeven","keystone","identity",True,"admin","http://Seven-127.0.0.1:5002" +"endpoint20_id","RegionSeven","keystone","identity",True,"internal","https://Seven-127.0.0.1:5001" +"endpoint21_id","RegionSeven","keystone","identity",True,"public","https://Seven-127.0.0.1:5000" +') + instances = described_class.instances + expect(instances).to have_array_of_instances_hash([ + {:name=>"RegionOne/keystone::identity", + :ensure=>:present, + :id=>"endpoint1_id,endpoint2_id,endpoint3_id", + :region=>"RegionOne", + :admin_url=>"http://One-127.0.0.1:5002", + :internal_url=>"https://One-127.0.0.1:5001", + :public_url=>"https://One-127.0.0.1:5000"}, + {:name=>"RegionTwo/keystone::identity", + :ensure=>:present, + :id=>"endpoint4_id,endpoint5_id,endpoint6_id", + :region=>"RegionTwo", + :admin_url=>"http://Two-127.0.0.1:5002", + :internal_url=>"https://Two-127.0.0.1:5001", + :public_url=>"https://Two-127.0.0.1:5000"}, + {:name=>"RegionThree/keystone::identity", + :ensure=>:present, + :id=>"endpoint7_id,endpoint8_id,endpoint9_id", + :region=>"RegionThree", + :admin_url=>"http://Three-127.0.0.1:5002", + :internal_url=>"https://Three-127.0.0.1:5001", + :public_url=>"https://Three-127.0.0.1:5000"}, + {:name=>"RegionFour/keystone::identity", + :ensure=>:present, + :id=>"endpoint10_id,endpoint11_id,endpoint12_id", + :region=>"RegionFour", + :admin_url=>"http://Four-127.0.0.1:5002", + :internal_url=>"https://Four-127.0.0.1:5001", + :public_url=>"https://Four-127.0.0.1:5000"}, + {:name=>"RegionFive/keystone::identity", + :ensure=>:present, + :id=>"endpoint13_id,endpoint14_id,endpoint15_id", + :region=>"RegionFive", + :admin_url=>"http://Five-127.0.0.1:5002", + :internal_url=>"https://Five-127.0.0.1:5001", + :public_url=>"https://Five-127.0.0.1:5000"}, + {:name=>"RegionSix/keystone::identity", + :ensure=>:present, + :id=>"endpoint16_id,endpoint17_id,endpoint18_id", + :region=>"RegionSix", + :admin_url=>"http://Six-127.0.0.1:5002", + :internal_url=>"https://Six-127.0.0.1:5001", + :public_url=>"https://Six-127.0.0.1:5000"}, + {:name=>"RegionSeven/keystone::identity", + :ensure=>:present, + :id=>"endpoint19_id,endpoint20_id,endpoint21_id", + :region=>"RegionSeven", + :admin_url=>"http://Seven-127.0.0.1:5002", + :internal_url=>"https://Seven-127.0.0.1:5001", + :public_url=>"https://Seven-127.0.0.1:5000"}]) + end end end diff --git a/tripleo/manifests/loadbalancer.pp b/tripleo/manifests/loadbalancer.pp index efca05700..f49ea1218 100644 --- a/tripleo/manifests/loadbalancer.pp +++ b/tripleo/manifests/loadbalancer.pp @@ -718,6 +718,10 @@ haproxy::listen { 'cinder': bind => $cinder_bind_opts, collect_exported => false, + mode => 'http', # Needed for http-request option + options => { + 'http-request' => ['set-header X-Forwarded-Proto https if { ssl_fc }'], + }, } haproxy::balancermember { 'cinder': listening_service => 'cinder', diff --git a/vswitch/.fixtures.yml b/vswitch/.fixtures.yml deleted file mode 100644 index ad2ac7c9b..000000000 --- a/vswitch/.fixtures.yml +++ /dev/null @@ -1,3 +0,0 @@ -fixtures: - symlinks: - "vswitch": "#{source_dir}" diff --git a/vswitch/.gitignore b/vswitch/.gitignore index 4dd84f06e..15c55efd0 100644 --- a/vswitch/.gitignore +++ b/vswitch/.gitignore @@ -8,3 +8,4 @@ coverage/ .idea/ *.swp *.iml +openstack/ diff --git a/vswitch/Gemfile b/vswitch/Gemfile index 6d4ce9a07..fc2214398 100644 --- a/vswitch/Gemfile +++ b/vswitch/Gemfile @@ -19,6 +19,7 @@ end group :system_tests do gem 'beaker-rspec', :require => 'false' gem 'beaker-puppet_install_helper', :require => 'false' + gem 'r10k', :require => 'false' end if facterversion = ENV['FACTER_GEM_VERSION'] diff --git a/vswitch/Rakefile b/vswitch/Rakefile index 7dcaeb962..9692ffdd2 100644 --- a/vswitch/Rakefile +++ b/vswitch/Rakefile @@ -1,6 +1,9 @@ require 'puppetlabs_spec_helper/rake_tasks' require 'puppet-lint/tasks/puppet-lint' require 'puppet-syntax/tasks/puppet-syntax' +require 'json' + +modname = JSON.parse(open('metadata.json').read)['name'].split('-')[1] PuppetSyntax.exclude_paths ||= [] PuppetSyntax.exclude_paths << "spec/fixtures/**/*" @@ -19,3 +22,64 @@ desc "Run acceptance tests" RSpec::Core::RakeTask.new(:acceptance) do |t| t.pattern = 'spec/acceptance' end + +Rake::Task[:spec_prep].clear +desc 'Create the fixtures directory' +task :spec_prep do + # Allow to test the module with custom dependencies + # like you could do with .fixtures file + if ENV['PUPPETFILE'] + puppetfile = ENV['PUPPETFILE'] + if ENV['GEM_HOME'] + gem_home = ENV['GEM_HOME'] + gem_bin_dir = "#{gem_home}" + '/bin/' + else + gem_bin_dir = '' + end + r10k = ['env'] + r10k += ["PUPPETFILE=#{puppetfile}"] + r10k += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] + r10k += ["#{gem_bin_dir}r10k"] + r10k += ['puppetfile', 'install', '-v'] + sh(*r10k) + else + # otherwise, use official OpenStack Puppetfile + zuul_ref = ENV['ZUUL_REF'] + zuul_branch = ENV['ZUUL_BRANCH'] + zuul_url = ENV['ZUUL_URL'] + repo = 'openstack/puppet-openstack-integration' + rm_rf(repo) + if File.exists?('/usr/zuul-env/bin/zuul-cloner') + zuul_clone_cmd = ['/usr/zuul-env/bin/zuul-cloner'] + zuul_clone_cmd += ['--cache-dir', '/opt/git'] + zuul_clone_cmd += ['--zuul-ref', "#{zuul_ref}"] + zuul_clone_cmd += ['--zuul-branch', "#{zuul_branch}"] + zuul_clone_cmd += ['--zuul-url', "#{zuul_url}"] + zuul_clone_cmd += ['git://git.openstack.org', "#{repo}"] + sh(*zuul_clone_cmd) + else + sh("git clone https://git.openstack.org/#{repo} -b stable/liberty #{repo}") + end + script = ['env'] + script += ["PUPPETFILE_DIR=#{Dir.pwd}/spec/fixtures/modules"] + script += ["ZUUL_REF=#{zuul_ref}"] + script += ["ZUUL_BRANCH=#{zuul_branch}"] + script += ["ZUUL_URL=#{zuul_url}"] + script += ['bash', "#{repo}/install_modules_unit.sh"] + sh(*script) + end + rm_rf("spec/fixtures/modules/#{modname}") + ln_s(Dir.pwd, "spec/fixtures/modules/#{modname}") + mkdir_p('spec/fixtures/manifests') + touch('spec/fixtures/manifests/site.pp') +end + +Rake::Task[:spec_clean].clear +desc 'Clean up the fixtures directory' +task :spec_clean do + rm_rf('spec/fixtures/modules') + rm_rf('openstack') + if File.zero?('spec/fixtures/manifests/site.pp') + rm_f('spec/fixtures/manifests/site.pp') + end +end From 889e89763f0a58d944277e14c047c5f48ec73a9b Mon Sep 17 00:00:00 2001 From: Lukas Bezdicka Date: Fri, 22 Jan 2016 11:45:18 +0100 Subject: [PATCH 18/35] Update mysql to bc1c44afa4b2e2075aa5992d1528d840bf554a34 bc1c44afa4b2e2075aa5992d1528d840bf554a34 Merge pull request #794 from DavidS/fm-4563-share-tmpdir 95841d3a8c4e2c0995b8fa74e5e0e0db25165325 (FM-4563) fix tmpdir to be shared across examples ecff1f3dd31bdb2ecd1605a3bbfe3d18a6dce4b0 Merge pull request #793 from DavidS/fm-4563-workaround-pup-5016 d89062faff9d62808c8b47f3404a3323f6153104 (FM-4563) skip idempotency tests on test cells that do have PUP-5016 unfixed 2de4597ecd5daafdbe8647206edae045531a400e (maint) refactor main acceptance suite a261a3d493b661057576292ec967f9d344b43359 (maint) fix logic for choosing rspec version 23672ca3265d129a4e8b97c1fc7d0e9aba05b318 Merge pull request #779 from jmcclell/patch-1 1bad8ae56a3d7ff8ec20845c42a2cafa13ad30aa Fixes edge-case with dropping pre-existing users with grants 0b3e310ba72c2ac77e86b6c2435b476200bfa1a1 Merge pull request #791 from roidelapluie/empty-string-as-parameter e30e0bc958761890ea4f06cdd3f1fc7242a00fe2 Re-Add the ability to set a empty string as option parameter 1cfddec3ff2baa12e5eb87606712f33a2b2b0473 Merge pull request #790 from cadeauthom/service-unmanaged b33f9c663eb6eb3d96756c975638c49f3230fa9e Move all dependencies to not have them in case of service unmanaged 5e7b999615bf99cc307b570c3eb27610d08df3c7 Merge pull request #784 from vicinus/master 3a49209a6f7f4df98e304840ef147e5496fc780f ensure if service restart to wait till mysql is up 58a55ade47cac1f4dc102f422408c0252a204cd0 Merge pull request #789 from elconas/fix_mysql576_pull_rebase3 7efc93c3c5b9c9f7893a2914e4249e85a8879407 Fixed new mysql_datadir provider on CentOS for MySQl 5.7.6 compatibility 45419fde6d7da5aa8914dcc3fc4629384c9db9e1 Merge pull request #787 from obi11235/master c353259fd95e58ef3ef4b2955294e2186b46efa2 Merge pull request #769 from gabriel403/master 681d4f856fd1ccda1a6152fdb6622f110827431b Ubuntu vivid should use systemd not upstart 3a29c5e5524024df72c3152022640a5e369faf86 Fixing error when disabling service management and the service does not exist 51950bf2d11b071cff21bf86379e245fd08f2070 Merge pull request #786 from DavidS/fix-576-rebase-of-763 bdf4d0f52dfc244d10bbd5b67efb791a39520ed2 Fixed MySQL 5.7.6++ compatibility 60393f7d4a42d96e67436249b940a2374bffbe77 Merge branch '3.6.x' into 'master' f06cc1269c0da682292d46e275d6da6442e73556 Merge pull request #572 from sharumpe/ticket/MODULES-1337-MariaDB_for_OpenSuSE_13.1 85e49164ad1173a37c222fe31f9b1e3f053965ef Merge pull request #780 from tphoney/release_3.6.2 9e2de7f36786c479d89e4ed5b5d3282b807f2d50 3.6.2 release prep 3a6a6c63580a71058aef74176d461547a3f0a660 Checking major release instead of specific release per @cmurphy. 47c1eb7386dcc39ac06b5947574ac07237b7a82c Merging with upstream changes from https://github.com/puppetlabs/puppetlabs-mysql.git d095721d7059aab4cf73a0f5a816b0ecd92ca5bd Update changelog 542c43eda5cbeecad814ad06fce2b006a2b5364e Improved user validation and munging 1d824777237a6b28884ee411750ebd8a24dc8d77 Merge pull request #756 from bmjen/3.6.1_prep f63b7d1f259206e7a721202dd7f9f59926c7d987 updates to 3.6.1 prep 58bb3b92d88cb9ec7d901db1f4336fa8c8aa507d Merge pull request #755 from mhaskel/3.6.1-prep f4b49f2c3582ed05b7f1556c2c1448acb8dd3912 3.6.1 prep daa4b625820f938d3e7a6ecc1ea77b5797110e33 Merge pull request #754 from mhaskel/ff_to_master d6bdd4cb3e00641acccd28faa522ef1ad4f38ce2 Compatibility with PE 3.3 9d5816ad5c338efe575adce51434b91ba654e6f1 Fixes improper use of function 'warn' in backup manifest of server. 54540324ea580ba711ea28b7de447485486f6ac1 Using mariadb in OpenSuSE >= 13.1. Change-Id: I3206871ca0a330e5cc05ba812ec96472f4d15ada --- Puppetfile | 2 +- mysql/CHANGELOG.md | 5 + mysql/Gemfile | 9 +- mysql/README.md | 14 ++ mysql/lib/puppet/provider/mysql.rb | 41 +++- .../puppet/provider/mysql_datadir/mysql.rb | 70 +++++++ .../lib/puppet/provider/mysql_grant/mysql.rb | 6 +- mysql/lib/puppet/provider/mysql_user/mysql.rb | 36 +++- mysql/lib/puppet/type/mysql_datadir.rb | 30 +++ mysql/manifests/params.pp | 36 +++- mysql/manifests/server/installdb.pp | 18 +- mysql/manifests/server/service.pp | 50 +++-- mysql/metadata.json | 4 +- mysql/spec/acceptance/mysql_db_spec.rb | 46 ++--- mysql/spec/acceptance/mysql_server_spec.rb | 44 ++--- mysql/spec/classes/mycnf_template_spec.rb | 5 + mysql/spec/classes/mysql_server_spec.rb | 14 +- mysql/spec/spec_helper_acceptance.rb | 16 ++ .../puppet/provider/mysql_user/mysql_spec.rb | 175 +++++++++++++++++- mysql/templates/my.cnf.erb | 2 +- 20 files changed, 523 insertions(+), 100 deletions(-) create mode 100644 mysql/lib/puppet/provider/mysql_datadir/mysql.rb create mode 100644 mysql/lib/puppet/type/mysql_datadir.rb diff --git a/Puppetfile b/Puppetfile index d7f22e981..40f1f8649 100644 --- a/Puppetfile +++ b/Puppetfile @@ -151,7 +151,7 @@ mod 'mongodb', :git => 'https://github.com/puppetlabs/puppetlabs-mongodb.git' mod 'mysql', - :commit => '7daa2979ef41545e9c9a3fbf7c670f2a7927afba', + :commit => 'bc1c44afa4b2e2075aa5992d1528d840bf554a34', :git => 'https://github.com/puppetlabs/puppetlabs-mysql.git' mod 'n1k_vsm', diff --git a/mysql/CHANGELOG.md b/mysql/CHANGELOG.md index 5cbe10369..00dd4635e 100644 --- a/mysql/CHANGELOG.md +++ b/mysql/CHANGELOG.md @@ -1,3 +1,8 @@ +## Supported Release 3.6.2 +###Summary + +Small release for support of newer PE versions. This increments the version of PE in the metadata.json file. + ## 2015-09-22 - Supported Release 3.6.1 ### Summary This is a security and bugfix release that fixes incorrect username truncation in the munge for the mysql_user type, incorrect function used in `mysql::server::backup` and fixes compatibility issues with PE 3.3.x. diff --git a/mysql/Gemfile b/mysql/Gemfile index 839dcc001..ee382e2e0 100644 --- a/mysql/Gemfile +++ b/mysql/Gemfile @@ -11,11 +11,12 @@ def location_for(place, fake_version = nil) end group :development, :unit_tests do - # rspec must be v2 for ruby 1.8.7 - if RUBY_VERSION >= '1.8.7' and RUBY_VERSION < '1.9' - gem 'rspec', '~> 2.0' + # rspec-core 3.1.7 is the last version to support ruby 1.8 + if RUBY_VERSION < '1.9' + gem 'rspec-core', '3.1.7' else - gem 'rspec-core', '3.1.7', :require => false + # newer version required to avoid BKR-537 + gem 'rspec-core', '>= 3.4' end gem 'puppetlabs_spec_helper', :require => false diff --git a/mysql/README.md b/mysql/README.md index 9961d6d4d..e0d767177 100644 --- a/mysql/README.md +++ b/mysql/README.md @@ -85,6 +85,8 @@ replicate-do-db = base1 replicate-do-db = base2 ~~~ +To implement version specific parameters you can use [mysqld-5.5] syntax which is only read by MySQL version 5.5. This allows one config for different versions of MySQL. + ### Creating a database To use `mysql::db` to create a database with a user and assign some privileges: @@ -181,6 +183,7 @@ mysql::db { 'mydb': #### Private classes * `mysql::server::install`: Installs packages. +* `mysql::server::installdb`: Implements setup of mysqld data directory (e.g. /var/lib/mysql) * `mysql::server::config`: Configures MYSQL. * `mysql::server::service`: Manages service. * `mysql::server::account_security`: Deletes default MySQL accounts. @@ -805,6 +808,17 @@ The name of the MySQL plugin to manage. The library file name. +#### `mysql_datadir` + +Initializes the MySQL data directory with version specific code. Pre MySQL 5.7.6 +it uses mysql_install_db. After MySQL 5.7.6 it uses mysqld --initialize-insecure. + +Insecure initialization is needed, as mysqld version 5.7 introduced "secure by default" mode. +This means MySQL generates a random password and writes it to STDOUT. This means puppet +can never accesss the database server afterwards, as no credencials are available. + +This type is an internal type and should not be called directly. + ### Facts #### `mysql_version` diff --git a/mysql/lib/puppet/provider/mysql.rb b/mysql/lib/puppet/provider/mysql.rb index 9b34ca00e..cf76f13af 100644 --- a/mysql/lib/puppet/provider/mysql.rb +++ b/mysql/lib/puppet/provider/mysql.rb @@ -2,7 +2,12 @@ class Puppet::Provider::Mysql < Puppet::Provider # Without initvars commands won't work. initvars + + # Make sure we find mysqld on CentOS + ENV['PATH']=ENV['PATH'] + ':/usr/libexec' + commands :mysql => 'mysql' + commands :mysqld => 'mysqld' commands :mysqladmin => 'mysqladmin' # Optional defaults file @@ -13,7 +18,41 @@ def self.defaults_file nil end end - + + def self.mysqld_type + # find the mysql "dialect" like mariadb / mysql etc. + mysqld_version_string.scan(/\s\(mariadb/i) { return "mariadb" } + mysqld_version_string.scan(/\s\(mysql/i) { return "mysql" } + mysqld_version_string.scan(/\s\(percona/i) { return "percona" } + nil + end + + def mysqld_type + self.class.mysqld_type + end + + def self.mysqld_version_string + # we cache the result ... + return @mysqld_version_string unless @mysqld_version_string.nil? + @mysqld_version_string = mysqld(['-V'].compact) + return @mysqld_version_string + end + + def mysqld_version_string + self.class.mysqld_version_string + end + + def self.mysqld_version + # note: be prepared for '5.7.6-rc-log' etc results + # versioncmp detects 5.7.6-log to be newer then 5.7.6 + # this is why we need the trimming. + mysqld_version_string.scan(/\d+\.\d+\.\d+/).first unless mysqld_version_string.nil? + end + + def mysqld_version + self.class.mysqld_version + end + def defaults_file self.class.defaults_file end diff --git a/mysql/lib/puppet/provider/mysql_datadir/mysql.rb b/mysql/lib/puppet/provider/mysql_datadir/mysql.rb new file mode 100644 index 000000000..c90d9753c --- /dev/null +++ b/mysql/lib/puppet/provider/mysql_datadir/mysql.rb @@ -0,0 +1,70 @@ +require File.expand_path(File.join(File.dirname(__FILE__), '..', 'mysql')) +Puppet::Type.type(:mysql_datadir).provide(:mysql, :parent => Puppet::Provider::Mysql) do + + desc 'manage data directories for mysql instances' + + initvars + + # Make sure we find mysqld on CentOS + ENV['PATH']=ENV['PATH'] + ':/usr/libexec' + + commands :mysqld => 'mysqld' + commands :mysql_install_db => 'mysql_install_db' + + def create + name = @resource[:name] + insecure = @resource.value(:insecure) || true + defaults_extra_file = @resource.value(:defaults_extra_file) + user = @resource.value(:user) || "mysql" + basedir = @resource.value(:basedir) || "/usr" + datadir = @resource.value(:datadir) || @resource[:name] + + unless defaults_extra_file.nil? + if File.exist?(defaults_extra_file) + defaults_extra_file="--defaults-extra-file=#{defaults_extra_file}" + else + raise ArgumentError, "Defaults-extra-file #{defaults_extra_file} is missing" + end + end + + if insecure == true + initialize="--initialize-insecure" + else + initialize="--initialize" + end + + if mysqld_version.nil? + debug("Installing MySQL data directory with mysql_install_db --basedir=#{basedir} #{defaults_extra_file} --datadir=#{datadir} --user=#{user}") + mysql_install_db(["--basedir=#{basedir}",defaults_extra_file, "--datadir=#{datadir}", "--user=#{user}"].compact) + else + if mysqld_type == "mysql" and Puppet::Util::Package.versioncmp(mysqld_version, '5.7.6') >= 0 + debug("Initializing MySQL data directory >= 5.7.6 with 'mysqld #{defaults_extra_file} #{initialize} --basedir=#{basedir} --datadir=#{datadir} --user=#{user}'") + mysqld([defaults_extra_file,initialize,"--basedir=#{basedir}","--datadir=#{datadir}", "--user=#{user}", "--log_error=/var/tmp/mysqld_initialize.log"].compact) + else + debug("Installing MySQL data directory with mysql_install_db --basedir=#{basedir} #{defaults_extra_file} --datadir=#{datadir} --user=#{user}") + mysql_install_db(["--basedir=#{basedir}",defaults_extra_file, "--datadir=#{datadir}", "--user=#{user}"].compact) + end + end + + exists? + end + + def destroy + name = @resource[:name] + raise ArgumentError, "ERROR: Resource can not be removed" + end + + def exists? + datadir = @resource[:datadir] + File.directory?("#{datadir}/mysql") + end + + ## + ## MySQL datadir properties + ## + + # Generates method for all properties of the property_hash + mk_resource_methods + +end + diff --git a/mysql/lib/puppet/provider/mysql_grant/mysql.rb b/mysql/lib/puppet/provider/mysql_grant/mysql.rb index e7a58f57f..8b19d2031 100644 --- a/mysql/lib/puppet/provider/mysql_grant/mysql.rb +++ b/mysql/lib/puppet/provider/mysql_grant/mysql.rb @@ -114,7 +114,11 @@ def revoke(user, table, revoke_privileges = ['ALL']) end def destroy - revoke(@property_hash[:user], @property_hash[:table]) + # if the user was dropped, it'll have been removed from the user hash + # as the grants are alraedy removed by the DROP statement + if self.class.users.include? @property_hash[:user] + revoke(@property_hash[:user], @property_hash[:table]) + end @property_hash.clear exists? ? (return false) : (return true) diff --git a/mysql/lib/puppet/provider/mysql_user/mysql.rb b/mysql/lib/puppet/provider/mysql_user/mysql.rb index 4ba194b7b..bc4014f41 100644 --- a/mysql/lib/puppet/provider/mysql_user/mysql.rb +++ b/mysql/lib/puppet/provider/mysql_user/mysql.rb @@ -12,7 +12,16 @@ def self.instances # To reduce the number of calls to MySQL we collect all the properties in # one big swoop. users.collect do |name| - query = "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, /*!50706 AUTHENTICATION_STRING AS */ PASSWORD /*!50508 , PLUGIN */ FROM mysql.user WHERE CONCAT(user, '@', host) = '#{name}'" + if mysqld_version.nil? + ## Default ... + query = "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, PASSWORD /*!50508 , PLUGIN */ FROM mysql.user WHERE CONCAT(user, '@', host) = '#{name}'" + else + if mysqld_type == "mysql" and Puppet::Util::Package.versioncmp(mysqld_version, '5.7.6') >= 0 + query = "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, AUTHENTICATION_STRING, PLUGIN FROM mysql.user WHERE CONCAT(user, '@', host) = '#{name}'" + else + query = "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, PASSWORD /*!50508 , PLUGIN */ FROM mysql.user WHERE CONCAT(user, '@', host) = '#{name}'" + end + end @max_user_connections, @max_connections_per_hour, @max_queries_per_hour, @max_updates_per_hour, @password, @plugin = mysql([defaults_file, "-NBe", query].compact).split(/\s/) @@ -51,7 +60,11 @@ def create # Use CREATE USER to be compatible with NO_AUTO_CREATE_USER sql_mode # This is also required if you want to specify a authentication plugin if !plugin.nil? - mysql([defaults_file, '-e', "CREATE USER '#{merged_name}' IDENTIFIED WITH '#{plugin}'"].compact) + if plugin == 'sha256_password' and !password_hash.nil? + mysql([defaults_file, '-e', "CREATE USER '#{merged_name}' IDENTIFIED WITH '#{plugin}' AS '#{password_hash}'"].compact) + else + mysql([defaults_file, '-e', "CREATE USER '#{merged_name}' IDENTIFIED WITH '#{plugin}'"].compact) + end @property_hash[:ensure] = :present @property_hash[:plugin] = plugin else @@ -89,7 +102,24 @@ def exists? def password_hash=(string) merged_name = self.class.cmd_user(@resource[:name]) - mysql([defaults_file, '-e', "SET PASSWORD FOR #{merged_name} = '#{string}'"].compact) + + # We have a fact for the mysql version ... + if mysqld_version.nil? + # default ... if mysqld_version does not work + mysql([defaults_file, '-e', "SET PASSWORD FOR #{merged_name} = '#{string}'"].compact) + else + # Version >= 5.7.6 (many password related changes) + if mysqld_type == "mysql" and Puppet::Util::Package.versioncmp(mysqld_version, '5.7.6') >= 0 + if string.match(/^\*/) + mysql([defaults_file, '-e', "ALTER USER #{merged_name} IDENTIFIED WITH mysql_native_password AS '#{string}'"].compact) + else + raise ArgumentError, "Only mysql_native_password (*ABCD...XXX) hashes are supported" + end + else + # older versions + mysql([defaults_file, '-e', "SET PASSWORD FOR #{merged_name} = '#{string}'"].compact) + end + end password_hash == string ? (return true) : (return false) end diff --git a/mysql/lib/puppet/type/mysql_datadir.rb b/mysql/lib/puppet/type/mysql_datadir.rb new file mode 100644 index 000000000..156b82766 --- /dev/null +++ b/mysql/lib/puppet/type/mysql_datadir.rb @@ -0,0 +1,30 @@ +Puppet::Type.newtype(:mysql_datadir) do + @doc = 'Manage MySQL datadirs with mysql_install_db OR mysqld (5.7.6 and above).' + + ensurable + + autorequire(:package) { 'mysql-server' } + + newparam(:datadir, :namevar => true) do + desc "The datadir name" + end + + newparam(:basedir) do + desc 'The basedir name, default /usr.' + newvalues(/^\//) + end + + newparam(:user) do + desc 'The user for the directory default mysql (name, not uid).' + end + + newparam(:defaults_extra_file) do + desc "MySQL defaults-extra-file with absolute path (*.cnf)." + newvalues(/^\/.*\.cnf$/) + end + + newparam(:insecure, :boolean => true) do + desc "Insecure initialization (needed for 5.7.6++)." + end + +end diff --git a/mysql/manifests/params.pp b/mysql/manifests/params.pp index 998cad401..a85187b3b 100644 --- a/mysql/manifests/params.pp +++ b/mysql/manifests/params.pp @@ -96,9 +96,17 @@ 'Suse': { case $::operatingsystem { 'OpenSuSE': { - $client_package_name = 'mysql-community-server-client' - $server_package_name = 'mysql-community-server' - $basedir = '/usr' + if versioncmp( $::operatingsystemmajrelease, '13' ) >= 0 { + $client_package_name = 'mariadb-client' + $server_package_name = 'mariadb' + # First service start fails if this is set. Runs fine without + # it being set, in any case. Leaving it as-is for the mysql. + $basedir = undef + } else { + $client_package_name = 'mysql-community-server-client' + $server_package_name = 'mysql-community-server' + $basedir = '/usr' + } } 'SLES','SLED': { if versioncmp($::operatingsystemrelease, '12') >= 0 { @@ -351,7 +359,11 @@ case $::operatingsystem { 'Ubuntu': { - $server_service_provider = upstart + if versioncmp($::operatingsystemmajrelease, '14.10') > 0 { + $server_service_provider = 'systemd' + } else { + $server_service_provider = 'upstart' + } } default: { $server_service_provider = undef @@ -368,6 +380,21 @@ 'log-error' => $mysql::params::log_error, 'socket' => $mysql::params::socket, }, + 'mysqld-5.0' => { + 'myisam-recover' => 'BACKUP', + }, + 'mysqld-5.1' => { + 'myisam-recover' => 'BACKUP', + }, + 'mysqld-5.5' => { + 'myisam-recover' => 'BACKUP', + }, + 'mysqld-5.6' => { + 'myisam-recover-options' => 'BACKUP', + }, + 'mysqld-5.7' => { + 'myisam-recover-options' => 'BACKUP', + }, 'mysqld' => { 'basedir' => $mysql::params::basedir, 'bind-address' => '127.0.0.1', @@ -378,7 +405,6 @@ 'max_allowed_packet' => '16M', 'max_binlog_size' => '100M', 'max_connections' => '151', - 'myisam_recover' => 'BACKUP', 'pid-file' => $mysql::params::pidfile, 'port' => '3306', 'query_cache_limit' => '1M', diff --git a/mysql/manifests/server/installdb.pp b/mysql/manifests/server/installdb.pp index de4772e6a..78e08f521 100644 --- a/mysql/manifests/server/installdb.pp +++ b/mysql/manifests/server/installdb.pp @@ -10,21 +10,21 @@ $config_file = $mysql::server::config_file if $mysql::server::manage_config_file { - $install_db_args = "--basedir=${basedir} --defaults-extra-file=${config_file} --datadir=${datadir} --user=${mysqluser}" + $_config_file=$config_file } else { - $install_db_args = "--basedir=${basedir} --datadir=${datadir} --user=${mysqluser}" + $_config_file=undef } - exec { 'mysql_install_db': - command => "mysql_install_db ${install_db_args}", - creates => "${datadir}/mysql", - logoutput => on_failure, - path => '/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin', - require => Package['mysql-server'], + mysql_datadir { $datadir: + ensure => 'present', + datadir => $datadir, + basedir => $basedir, + user => $mysqluser, + defaults_extra_file => $_config_file, } if $mysql::server::restart { - Exec['mysql_install_db'] { + Mysql_datadir[$datadir] { notify => Class['mysql::server::service'], } } diff --git a/mysql/manifests/server/service.pp b/mysql/manifests/server/service.pp index 6b9a05395..9aa60f1b2 100644 --- a/mysql/manifests/server/service.pp +++ b/mysql/manifests/server/service.pp @@ -26,25 +26,41 @@ } } - service { 'mysqld': - ensure => $service_ensure, - name => $mysql::server::service_name, - enable => $mysql::server::real_service_enabled, - provider => $mysql::server::service_provider, - } + if $mysql::server::real_service_manage { + service { 'mysqld': + ensure => $service_ensure, + name => $mysql::server::service_name, + enable => $mysql::server::real_service_enabled, + provider => $mysql::server::service_provider, + } - # only establish ordering between service and package if - # we're managing the package. - if $mysql::server::package_manage { - Service['mysqld'] { - require => Package['mysql-server'], + # only establish ordering between service and package if + # we're managing the package. + if $mysql::server::package_manage { + Service['mysqld'] { + require => Package['mysql-server'], + } } - } - # only establish ordering between config file and service if - # we're managing the config file. - if $mysql::server::manage_config_file { - File['mysql-config-file'] -> Service['mysqld'] - } + # only establish ordering between config file and service if + # we're managing the config file. + if $mysql::server::manage_config_file { + File['mysql-config-file'] -> Service['mysqld'] + } + if $mysql::server::override_options and $mysql::server::override_options['mysqld'] and $mysql::server::override_options['mysqld']['socket'] { + $mysqlsocket = $mysql::server::override_options['mysqld']['socket'] + } else { + $mysqlsocket = $options['mysqld']['socket'] + } + + exec { 'wait_for_mysql_socket_to_open': + command => "test -S ${mysqlsocket}", + unless => "test -S ${mysqlsocket}", + tries => '3', + try_sleep => '10', + require => Service['mysqld'], + path => '/bin:/usr/bin', + } + } } diff --git a/mysql/metadata.json b/mysql/metadata.json index 9bb1bb127..523e2ee72 100644 --- a/mysql/metadata.json +++ b/mysql/metadata.json @@ -1,6 +1,6 @@ { "name": "puppetlabs-mysql", - "version": "3.6.1", + "version": "3.6.2", "author": "Puppet Labs", "summary": "Installs, configures, and manages the MySQL service.", "license": "Apache-2.0", @@ -70,7 +70,7 @@ "requirements": [ { "name": "pe", - "version_requirement": ">= 3.0.0 < 2015.3.0" + "version_requirement": ">= 3.0.0 < 2015.4.0" }, { "name": "puppet", diff --git a/mysql/spec/acceptance/mysql_db_spec.rb b/mysql/spec/acceptance/mysql_db_spec.rb index 88ded3aec..8c571608e 100644 --- a/mysql/spec/acceptance/mysql_db_spec.rb +++ b/mysql/spec/acceptance/mysql_db_spec.rb @@ -2,28 +2,26 @@ describe 'mysql::db define' do describe 'creating a database' do - # Using puppet_apply as a helper - it 'should work with no errors' do - pp = <<-EOS + let(:pp) do + <<-EOS class { 'mysql::server': root_password => 'password' } mysql::db { 'spec1': user => 'root1', password => 'password', } EOS + end + it_behaves_like "a idempotent resource" - # Run it twice and test for idempotency - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) - - expect(shell("mysql -e 'show databases;'|grep spec1").exit_code).to be_zero + describe command("mysql -e 'show databases;'") do + its(:exit_status) { is_expected.to eq 0 } + its(:stdout) { is_expected.to match /^spec1$/ } end end describe 'creating a database with post-sql' do - # Using puppet_apply as a helper - it 'should work with no errors' do - pp = <<-EOS + let(:pp) do + <<-EOS class { 'mysql::server': override_options => { 'root_password' => 'password' } } file { '/tmp/spec.sql': ensure => file, @@ -36,21 +34,19 @@ class { 'mysql::server': override_options => { 'root_password' => 'password' } } sql => '/tmp/spec.sql', } EOS - - # Run it twice and test for idempotency - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) end + it_behaves_like "a idempotent resource" - it 'should have the table' do - expect(shell("mysql -e 'show tables;' spec2|grep table1").exit_code).to be_zero + describe command("mysql -e 'show tables;' spec2") do + its(:exit_status) { is_expected.to eq 0 } + its(:stdout) { is_expected.to match /^table1$/ } end end describe 'creating a database with dbname parameter' do - # Using puppet_apply as a helper - it 'should work with no errors' do - pp = <<-EOS + let(:check_command) { " | grep realdb" } + let(:pp) do + <<-EOS class { 'mysql::server': override_options => { 'root_password' => 'password' } } mysql::db { 'spec1': user => 'root1', @@ -58,14 +54,12 @@ class { 'mysql::server': override_options => { 'root_password' => 'password' } } dbname => 'realdb', } EOS - - # Run it twice and test for idempotency - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) end + it_behaves_like "a idempotent resource" - it 'should have the database named realdb' do - expect(shell("mysql -e 'show databases;'|grep realdb").exit_code).to be_zero + describe command("mysql -e 'show databases;'") do + its(:exit_status) { is_expected.to eq 0 } + its(:stdout) { is_expected.to match /^realdb$/ } end end end diff --git a/mysql/spec/acceptance/mysql_server_spec.rb b/mysql/spec/acceptance/mysql_server_spec.rb index e19be5519..06ea0ba04 100644 --- a/mysql/spec/acceptance/mysql_server_spec.rb +++ b/mysql/spec/acceptance/mysql_server_spec.rb @@ -1,15 +1,15 @@ require 'spec_helper_acceptance' describe 'mysql class' do - - describe 'running puppet code' do - # Using puppet_apply as a helper - it 'should work with no errors' do - tmpdir = default.tmpdir('mysql') - pp = <<-EOS + describe 'advanced config' do + before(:all) do + @tmpdir = default.tmpdir('mysql') + end + let(:pp) do + <<-EOS class { 'mysql::server': - config_file => '#{tmpdir}/my.cnf', - includedir => '#{tmpdir}/include', + config_file => '#{@tmpdir}/my.cnf', + includedir => '#{@tmpdir}/include', manage_config_file => 'true', override_options => { 'mysqld' => { 'key_buffer_size' => '32M' }}, package_ensure => 'present', @@ -46,39 +46,33 @@ class { 'mysql::server': } } EOS - - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) end + + it_behaves_like "a idempotent resource" end - describe 'configuration needed for syslog' do - it 'should work with no errors' do - pp = <<-EOS + describe 'syslog configuration' do + let(:pp) do + <<-EOS class { 'mysql::server': override_options => { 'mysqld' => { 'log-error' => undef }, 'mysqld_safe' => { 'log-error' => false, 'syslog' => true }}, } EOS - - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) end + + it_behaves_like "a idempotent resource" end - describe 'when changing the password' do + context 'when changing the password' do let(:password) { 'THE NEW SECRET' } - let(:manifest) { "class { 'mysql::server': root_password => '#{password}' }" } + let(:pp) { "class { 'mysql::server': root_password => '#{password}' }" } it 'should not display the password' do - result = apply_manifest(manifest, :expect_changes => true) + result = apply_manifest(pp, :catch_failures => true) # this does not actually prove anything, as show_diff in the puppet config defaults to false. expect(result.stdout).not_to match /#{password}/ end - it 'should be idempotent' do - result = apply_manifest(manifest, :catch_changes => true) - end - + it_behaves_like "a idempotent resource" end - end diff --git a/mysql/spec/classes/mycnf_template_spec.rb b/mysql/spec/classes/mycnf_template_spec.rb index 6446c90f9..9c0c433e1 100644 --- a/mysql/spec/classes/mycnf_template_spec.rb +++ b/mysql/spec/classes/mycnf_template_spec.rb @@ -29,6 +29,11 @@ end end + describe 'skip-name-resolve set to an empty string' do + let(:params) {{ :override_options => { 'mysqld' => { 'skip-name-resolve' => '' }}}} + it { is_expected.to contain_file('mysql-config-file').with_content(/^skip-name-resolve$/) } + end + describe 'ssl set to true' do let(:params) {{ :override_options => { 'mysqld' => { 'ssl' => true }}}} it { is_expected.to contain_file('mysql-config-file').with_content(/ssl/) } diff --git a/mysql/spec/classes/mysql_server_spec.rb b/mysql/spec/classes/mysql_server_spec.rb index 0625895ee..6f682888a 100644 --- a/mysql/spec/classes/mysql_server_spec.rb +++ b/mysql/spec/classes/mysql_server_spec.rb @@ -43,7 +43,7 @@ end context 'with datadir overridden' do let(:params) {{ :override_options => { 'mysqld' => { 'datadir' => '/tmp' }} }} - it { is_expected.to contain_exec('mysql_install_db') } + it { is_expected.to contain_mysql_datadir('/tmp') } end end @@ -68,6 +68,18 @@ :ensure => :stopped }) end + context 'with package_manage set to true' do + let(:params) {{ :package_manage => true }} + it { is_expected.to contain_package('mysql-server') } + end + context 'with package_manage set to false' do + let(:params) {{ :package_manage => false }} + it { is_expected.not_to contain_package('mysql-server') } + end + context 'with datadir overridden' do + let(:params) {{ :override_options => { 'mysqld' => { 'datadir' => '/tmp' }} }} + it { is_expected.to contain_mysql_datadir('/tmp') } + end end context 'with log-error overridden' do let(:params) {{ :override_options => { 'mysqld' => { 'log-error' => '/tmp/error.log' }} }} diff --git a/mysql/spec/spec_helper_acceptance.rb b/mysql/spec/spec_helper_acceptance.rb index d23201381..d25f6a8b1 100644 --- a/mysql/spec/spec_helper_acceptance.rb +++ b/mysql/spec/spec_helper_acceptance.rb @@ -12,6 +12,12 @@ # Readable test descriptions c.formatter = :documentation + # detect the situation where PUP-5016 is triggered and skip the idempotency tests in that case + # also note how fact('puppetversion') is not available because of PUP-4359 + if fact('osfamily') == 'Debian' && fact('operatingsystemmajrelease') == '8' && shell('puppet --version').stdout =~ /^4\.2/ + c.filter_run_excluding :skip_pup_5016 => true + end + # Configure all nodes in nodeset c.before :suite do # Install module and dependencies @@ -47,3 +53,13 @@ end end end + +shared_examples "a idempotent resource" do + it 'should apply with no errors' do + apply_manifest(pp, :catch_failures => true) + end + + it 'should apply a second time without changes', :skip_pup_5016 do + apply_manifest(pp, :catch_changes => true) + end +end diff --git a/mysql/spec/unit/puppet/provider/mysql_user/mysql_spec.rb b/mysql/spec/unit/puppet/provider/mysql_user/mysql_spec.rb index aec759334..7eff67fec 100644 --- a/mysql/spec/unit/puppet/provider/mysql_user/mysql_spec.rb +++ b/mysql/spec/unit/puppet/provider/mysql_user/mysql_spec.rb @@ -1,6 +1,47 @@ require 'spec_helper' describe Puppet::Type.type(:mysql_user).provider(:mysql) do + + # Output of mysqld -V + mysql_version_string_hash = { + 'mysql-5.5' => + { + :version => '5.5.46', + :string => '/usr/sbin/mysqld Ver 5.5.46-log for Linux on x86_64 (MySQL Community Server (GPL))', + :mysql_type => 'mysql', + }, + 'mysql-5.6' => + { + :version => '5.6.27', + :string => '/usr/sbin/mysqld Ver 5.6.27 for Linux on x86_64 (MySQL Community Server (GPL))', + :mysql_type => 'mysql', + }, + 'mysql-5.7.1' => + { + :version => '5.7.1', + :string => '/usr/sbin/mysqld Ver 5.7.1 for Linux on x86_64 (MySQL Community Server (GPL))', + :mysql_type => 'mysql', + }, + 'mysql-5.7.6' => + { + :version => '5.7.8', + :string => '/usr/sbin/mysqld Ver 5.7.8-rc for Linux on x86_64 (MySQL Community Server (GPL))', + :mysql_type => 'mysql', + }, + 'mariadb-10.0' => + { + :version => '10.0.21', + :string => '/usr/sbin/mysqld Ver 10.0.21-MariaDB for Linux on x86_64 (MariaDB Server)', + :mysql_type => 'mariadb', + }, + 'percona-5.5' => + { + :version => '5.5.39', + :string => 'mysqld Ver 5.5.39-36.0-55 for Linux on x86_64 (Percona XtraDB Cluster (GPL), Release rel36.0, Revision 824, WSREP version 25.11, wsrep_25.11.r4023)', + :mysql_type => 'percona', + }, + } + let(:defaults_file) { '--defaults-extra-file=/root/.my.cnf' } let(:newhash) { '*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5' } @@ -35,19 +76,72 @@ # Set up the stubs for an instances call. Facter.stubs(:value).with(:root_home).returns('/root') Facter.stubs(:value).with(:mysql_version).returns('5.6.24') + provider.class.instance_variable_set(:@mysqld_version_string, '5.6.24') Puppet::Util.stubs(:which).with('mysql').returns('/usr/bin/mysql') + Puppet::Util.stubs(:which).with('mysqld').returns('/usr/sbin/mysqld') File.stubs(:file?).with('/root/.my.cnf').returns(true) provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT CONCAT(User, '@',Host) AS User FROM mysql.user"]).returns('joe@localhost') - provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, /*!50706 AUTHENTICATION_STRING AS */ PASSWORD /*!50508 , PLUGIN */ FROM mysql.user WHERE CONCAT(user, '@', host) = 'joe@localhost'"]).returns('10 10 10 10 *6C8989366EAF75BB670AD8EA7A7FC1176A95CEF4') + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, PASSWORD /*!50508 , PLUGIN */ FROM mysql.user WHERE CONCAT(user, '@', host) = 'joe@localhost'"]).returns('10 10 10 10 *6C8989366EAF75BB670AD8EA7A7FC1176A95CEF4') end let(:instance) { provider.class.instances.first } describe 'self.instances' do - it 'returns an array of users' do + it 'returns an array of users MySQL 5.5' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['mysql-5.5'][:string]) + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT CONCAT(User, '@',Host) AS User FROM mysql.user"]).returns(raw_users) + parsed_users.each do |user| + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, PASSWORD /*!50508 , PLUGIN */ FROM mysql.user WHERE CONCAT(user, '@', host) = '#{user}'"]).returns('10 10 10 10 ') + end + + usernames = provider.class.instances.collect {|x| x.name } + expect(parsed_users).to match_array(usernames) + end + it 'returns an array of users MySQL 5.6' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['mysql-5.6'][:string]) + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT CONCAT(User, '@',Host) AS User FROM mysql.user"]).returns(raw_users) + parsed_users.each do |user| + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, PASSWORD /*!50508 , PLUGIN */ FROM mysql.user WHERE CONCAT(user, '@', host) = '#{user}'"]).returns('10 10 10 10 ') + end + + usernames = provider.class.instances.collect {|x| x.name } + expect(parsed_users).to match_array(usernames) + end + it 'returns an array of users MySQL >= 5.7.0 < 5.7.6' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['mysql-5.7.1'][:string]) + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT CONCAT(User, '@',Host) AS User FROM mysql.user"]).returns(raw_users) + parsed_users.each do |user| + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, PASSWORD /*!50508 , PLUGIN */ FROM mysql.user WHERE CONCAT(user, '@', host) = '#{user}'"]).returns('10 10 10 10 ') + end + + usernames = provider.class.instances.collect {|x| x.name } + expect(parsed_users).to match_array(usernames) + end + it 'returns an array of users MySQL >= 5.7.6' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['mysql-5.7.6'][:string]) + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT CONCAT(User, '@',Host) AS User FROM mysql.user"]).returns(raw_users) + parsed_users.each do |user| + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, AUTHENTICATION_STRING, PLUGIN FROM mysql.user WHERE CONCAT(user, '@', host) = '#{user}'"]).returns('10 10 10 10 ') + end + + usernames = provider.class.instances.collect {|x| x.name } + expect(parsed_users).to match_array(usernames) + end + it 'returns an array of users mariadb 10.0' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['mariadb-10.0'][:string]) + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT CONCAT(User, '@',Host) AS User FROM mysql.user"]).returns(raw_users) + parsed_users.each do |user| + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, PASSWORD /*!50508 , PLUGIN */ FROM mysql.user WHERE CONCAT(user, '@', host) = '#{user}'"]).returns('10 10 10 10 ') + end + + usernames = provider.class.instances.collect {|x| x.name } + expect(parsed_users).to match_array(usernames) + end + it 'returns an array of users percona 5.5' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['percona-5.5'][:string]) provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT CONCAT(User, '@',Host) AS User FROM mysql.user"]).returns(raw_users) parsed_users.each do |user| - provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, /*!50706 AUTHENTICATION_STRING AS */ PASSWORD /*!50508 , PLUGIN */ FROM mysql.user WHERE CONCAT(user, '@', host) = '#{user}'"]).returns('10 10 10 10 ') + provider.class.stubs(:mysql).with([defaults_file, '-NBe', "SELECT MAX_USER_CONNECTIONS, MAX_CONNECTIONS, MAX_QUESTIONS, MAX_UPDATES, PASSWORD /*!50508 , PLUGIN */ FROM mysql.user WHERE CONCAT(user, '@', host) = '#{user}'"]).returns('10 10 10 10 ') end usernames = provider.class.instances.collect {|x| x.name } @@ -55,6 +149,19 @@ end end + describe 'mysql version and type detection' do + mysql_version_string_hash.each do |name,line| + version=line[:version] + string=line[:string] + mysql_type=line[:mysql_type] + it "detects type '#{mysql_type}' with version '#{version}'" do + provider.class.instance_variable_set(:@mysqld_version_string, string) + expect(provider.mysqld_version).to eq(version) + expect(provider.mysqld_type).to eq(mysql_type) + end + end + end + describe 'self.prefetch' do it 'exists' do provider.class.instances @@ -85,6 +192,30 @@ end end + describe 'self.mysqld_version' do + it 'queries mysql if unset' do + provider.class.instance_variable_set(:@mysqld_version_string, nil) + provider.class.expects(:mysqld).with(['-V']) + expect(provider.mysqld_version).to be_nil + end + it 'returns 5.7.6 for "mysqld Ver 5.7.6 for Linux on x86_64 (MySQL Community Server (GPL))"' do + provider.class.instance_variable_set(:@mysqld_version_string, 'mysqld Ver 5.7.6 for Linux on x86_64 (MySQL Community Server (GPL))') + expect(provider.mysqld_version).to eq '5.7.6' + end + it 'returns 5.7.6 for "mysqld Ver 5.7.6-rc for Linux on x86_64 (MySQL Community Server (GPL))"' do + provider.class.instance_variable_set(:@mysqld_version_string, 'mysqld Ver 5.7.6-rc for Linux on x86_64 (MySQL Community Server (GPL))') + expect(provider.mysqld_version).to eq '5.7.6' + end + it 'detects >= 5.7.6 for 5.7.7-log' do + provider.class.instance_variable_set(:@mysqld_version_string, 'mysqld Ver 5.7.7-log for Linux on x86_64 (MySQL Community Server (GPL))') + expect(Puppet::Util::Package.versioncmp(provider.mysqld_version, '5.7.6')).to be >= 0 + end + it 'detects < 5.7.6 for 5.7.5-log' do + provider.class.instance_variable_set(:@mysqld_version_string, 'mysqld Ver 5.7.5-log for Linux on x86_64 (MySQL Community Server (GPL))') + expect(Puppet::Util::Package.versioncmp(provider.mysqld_version, '5.7.6')).to be < 0 + end + end + describe 'self.defaults_file' do it 'sets --defaults-extra-file' do File.stubs(:file?).with('/root/.my.cnf').returns(true) @@ -103,7 +234,43 @@ end describe 'password_hash=' do - it 'changes the hash' do + it 'changes the hash mysql 5.5' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['mysql-5.5'][:string]) + provider.expects(:mysql).with([defaults_file, '-e', "SET PASSWORD FOR 'joe'@'localhost' = '*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5'"]).returns('0') + + provider.expects(:password_hash).returns('*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5') + provider.password_hash=('*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5') + end + it 'changes the hash mysql 5.6' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['mysql-5.6'][:string]) + provider.expects(:mysql).with([defaults_file, '-e', "SET PASSWORD FOR 'joe'@'localhost' = '*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5'"]).returns('0') + + provider.expects(:password_hash).returns('*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5') + provider.password_hash=('*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5') + end + it 'changes the hash mysql < 5.7.6' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['mysql-5.7.1'][:string]) + provider.expects(:mysql).with([defaults_file, '-e', "SET PASSWORD FOR 'joe'@'localhost' = '*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5'"]).returns('0') + + provider.expects(:password_hash).returns('*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5') + provider.password_hash=('*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5') + end + it 'changes the hash MySQL >= 5.7.6' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['mysql-5.7.6'][:string]) + provider.expects(:mysql).with([defaults_file, '-e', "ALTER USER 'joe'@'localhost' IDENTIFIED WITH mysql_native_password AS '*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5'"]).returns('0') + + provider.expects(:password_hash).returns('*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5') + provider.password_hash=('*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5') + end + it 'changes the hash mariadb-10.0' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['mariadb-10.0'][:string]) + provider.expects(:mysql).with([defaults_file, '-e', "SET PASSWORD FOR 'joe'@'localhost' = '*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5'"]).returns('0') + + provider.expects(:password_hash).returns('*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5') + provider.password_hash=('*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5') + end + it 'changes the hash percona-5.5' do + provider.class.instance_variable_set(:@mysqld_version_string, mysql_version_string_hash['percona-5.5'][:string]) provider.expects(:mysql).with([defaults_file, '-e', "SET PASSWORD FOR 'joe'@'localhost' = '*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5'"]).returns('0') provider.expects(:password_hash).returns('*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF5') diff --git a/mysql/templates/my.cnf.erb b/mysql/templates/my.cnf.erb index 7d832d978..7d1f1486b 100644 --- a/mysql/templates/my.cnf.erb +++ b/mysql/templates/my.cnf.erb @@ -6,7 +6,7 @@ <% v.sort.map do |ki, vi| -%> <% if ki == 'ssl-disable' or (ki =~ /^ssl/ and v['ssl-disable'] == true) -%> <% next %> -<% elsif vi == true or v == '' -%> +<% elsif vi == true or vi == '' -%> <%= ki %> <% elsif vi.is_a?(Array) -%> <% vi.each do |vii| -%> From 33b8dbdd35d2c13452d9480eada1398deb6c41c1 Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Thu, 28 Jan 2016 10:15:19 -0500 Subject: [PATCH 19/35] Update ceph to 45c81859a5ff37c6cbbce0d260801e9f25354515 45c81859a5ff37c6cbbce0d260801e9f25354515 Merge "Add support for setting globally ms_bind_ipv6" 6cf8a4ea88ef031cab4b18c2a666badd779948db Merge "Adds public_addr setting." 3d7d647789455e36993fdd97c68408093cc2b16c Add support for setting globally ms_bind_ipv6 e26cb47661703e30553bd6fc6cb79f4f932419f4 drop mock dependency 5252808a9b6b5e883c92f1e42155a95d467ecbc9 Adds public_addr setting. d387b0725f8eb01f2e9b572f7e3c22d5e00c67d9 Merge "Added default timeout for exec resources" 749a2a8026d4f5d3cc54c119b7be8dab12595eb9 Merge "fix mon creation to be more idempotent" 8fd97b5530c21ebfc82044fdeb368097cf72134a Merge "Be more specific with CentOS7 package exclude" 6a03609cce9f7b1f379230ef01d844723cf051da Added default timeout for exec resources 5096c97ddfbd4b491eb6050db41df04a93c0a8b9 fix mon creation to be more idempotent 618a5b2553590be76fe6a4c792b6c23d0a04bf0d Be more specific with CentOS7 package exclude Change-Id: Id79b9cb106e33908ba7837ee0e698e8c463f03ed --- Puppetfile | 2 +- ceph/Gemfile | 1 - ceph/manifests/init.pp | 10 + ceph/manifests/mon.pp | 19 +- ceph/manifests/osd.pp | 6 + ceph/manifests/params.pp | 3 + ceph/manifests/pool.pp | 9 + ceph/manifests/profile/base.pp | 1 + ceph/manifests/profile/params.pp | 4 + ceph/manifests/repo.pp | 6 +- ceph/spec/classes/ceph_init_spec.rb | 6 + ceph/spec/classes/ceph_repo_spec.rb | 557 ++++++++++++++++++++++++++++ ceph/spec/defines/ceph_mon_spec.rb | 12 +- 13 files changed, 630 insertions(+), 6 deletions(-) diff --git a/Puppetfile b/Puppetfile index 40f1f8649..c89537e29 100644 --- a/Puppetfile +++ b/Puppetfile @@ -19,7 +19,7 @@ mod 'ceilometer', :git => 'https://github.com/openstack/puppet-ceilometer.git' mod 'ceph', - :commit => 'e4b05caf4825af51f679f0618229dce4c3388a5f', + :commit => '45c81859a5ff37c6cbbce0d260801e9f25354515', :git => 'https://github.com/stackforge/puppet-ceph.git' mod 'certmonger', diff --git a/ceph/Gemfile b/ceph/Gemfile index e3e4a8158..b5da2527b 100644 --- a/ceph/Gemfile +++ b/ceph/Gemfile @@ -17,7 +17,6 @@ group :development, :test do gem 'puppet-lint-numericvariable' gem 'json' - gem 'webmock' gem 'minitest', :require => false gem 'test', :require => false gem 'test-unit', :require => false diff --git a/ceph/manifests/init.pp b/ceph/manifests/init.pp index e319ae7a1..d27907db0 100644 --- a/ceph/manifests/init.pp +++ b/ceph/manifests/init.pp @@ -71,6 +71,9 @@ # individually through ceph::mon. # Optional. String like e.g. 'a, b, c'. # +# [*ms_bind_ipv6*] Enables Ceph daemons to bind to IPv6 addresses. +# Optional. Boolean. Default provided by Ceph. +# # [*require_signatures*] If Ceph requires signatures on all # message traffic (client<->cluster and between cluster daemons). # Optional. Boolean. Default provided by Ceph. @@ -92,6 +95,9 @@ # [*public_network*] The address of the public network. # Optional. {public-network-ip/netmask} # +# [*public_addr*] The address of the node (on public network.) +# Optional. {public-network-ip} +# class ceph ( $fsid, $ensure = present, @@ -107,12 +113,14 @@ $mon_osd_nearfull_ratio = undef, $mon_initial_members = undef, $mon_host = undef, + $ms_bind_ipv6 = undef, $require_signatures = undef, $cluster_require_signatures = undef, $service_require_signatures = undef, $sign_messages = undef, $cluster_network = undef, $public_network = undef, + $public_addr = undef, ) { include ::ceph::params @@ -137,12 +145,14 @@ 'global/mon_osd_nearfull_ratio': value => $mon_osd_nearfull_ratio; 'global/mon_initial_members': value => $mon_initial_members; 'global/mon_host': value => $mon_host; + 'global/ms_bind_ipv6': value => $ms_bind_ipv6; 'global/require_signatures': value => $require_signatures; 'global/cluster_require_signatures': value => $cluster_require_signatures; 'global/service_require_signatures': value => $service_require_signatures; 'global/sign_messages': value => $sign_messages; 'global/cluster_network': value => $cluster_network; 'global/public_network': value => $public_network; + 'global/public_addr': value => $public_addr; 'osd/osd_journal_size': value => $osd_journal_size; } diff --git a/ceph/manifests/mon.pp b/ceph/manifests/mon.pp index 190f10742..f1429e7df 100644 --- a/ceph/manifests/mon.pp +++ b/ceph/manifests/mon.pp @@ -51,6 +51,9 @@ # [*keyring*] Path of the [mon.] keyring file # Optional. $key and $keyring are mutually exclusive. # +# [*exec_timeout*] The default exec resource timeout, in seconds +# Optional. Defaults to $::ceph::params::exec_timeout +# define ceph::mon ( $ensure = present, $public_addr = undef, @@ -58,6 +61,7 @@ $authentication_type = 'cephx', $key = undef, $keyring = undef, + $exec_timeout = $::ceph::params::exec_timeout, ) { # a puppet name translates into a ceph id, the meaning is different @@ -133,6 +137,9 @@ command => "/bin/true # comment to satisfy puppet syntax requirements set -ex touch /etc/ceph/${cluster_name}.client.admin.keyring", + unless => "/bin/true # comment to satisfy puppet syntax requirements +set -ex +test -e /etc/ceph/${cluster_name}.client.admin.keyring", } -> exec { $ceph_mkfs: @@ -151,21 +158,30 @@ rm -fr \$mon_data fi fi +", + unless => "/bin/true # comment to satisfy puppet syntax requirements +set -ex +mon_data=\$(ceph-mon ${cluster_option} --id ${id} --show-config-value mon_data) +test -d \$mon_data ", logoutput => true, + timeout => $exec_timeout, } -> service { $mon_service: ensure => running, } - if $authentication_type == 'cephx' { if $key { Exec[$ceph_mkfs] -> Exec["rm-keyring-${id}"] exec { "rm-keyring-${id}": command => "/bin/rm ${keyring_path}", + unless => "/bin/true # comment to satisfy puppet syntax requirements +set -ex +test ! -e ${keyring_path} +", } } } @@ -188,6 +204,7 @@ test ! -d \$mon_data ", logoutput => true, + timeout => $exec_timeout, } -> Package<| tag == 'ceph' |> } } diff --git a/ceph/manifests/osd.pp b/ceph/manifests/osd.pp index 83583ada3..356442c6d 100644 --- a/ceph/manifests/osd.pp +++ b/ceph/manifests/osd.pp @@ -38,10 +38,14 @@ # [*cluster*] The ceph cluster # Optional. Same default as ceph. # +# [*exec_timeout*] The default exec resource timeout, in seconds +# Optional. Defaults to $::ceph::params::exec_timeout +# define ceph::osd ( $ensure = present, $journal = undef, $cluster = undef, + $exec_timeout = $::ceph::params::exec_timeout, ) { $data = $name @@ -76,6 +80,7 @@ ls -l /var/lib/ceph/osd/${cluster_name}-* | grep ' ${data}\$' ", logoutput => true, + timeout => $exec_timeout, } Exec[$ceph_prepare] -> Exec[$ceph_activate] @@ -135,6 +140,7 @@ fi ", logoutput => true, + timeout => $exec_timeout, } -> Ceph::Mon<| ensure == absent |> } diff --git a/ceph/manifests/params.pp b/ceph/manifests/params.pp index 4d2c50a2f..999da3758 100644 --- a/ceph/manifests/params.pp +++ b/ceph/manifests/params.pp @@ -22,6 +22,8 @@ # should be considered to be constant # # === Parameters: +# [*exec_timeout*] The default exec resource timeout, in seconds +# Optional. Defaults to 600 # # [*packages*] The ceph package name # Optional. Defaults to 'ceph' @@ -30,6 +32,7 @@ # Optional. Defaults to '/tmp/radosgw.sock' # class ceph::params ( + $exec_timeout = 600, $packages = 'ceph', # just provide the minimum per default $rgw_socket_path = '/tmp/radosgw.sock', ) { diff --git a/ceph/manifests/pool.pp b/ceph/manifests/pool.pp index fd2edf595..a30d21d5e 100644 --- a/ceph/manifests/pool.pp +++ b/ceph/manifests/pool.pp @@ -43,11 +43,15 @@ # Optional. Default is undef. # Increase or decrease the replica level of a pool. # +# [*exec_timeout*] The default exec resource timeout, in seconds +# Optional. Defaults to $::ceph::params::exec_timeout +# define ceph::pool ( $ensure = present, $pg_num = 64, $pgp_num = undef, $size = undef, + $exec_timeout = $::ceph::params::exec_timeout, ) { if $ensure == present { @@ -62,6 +66,7 @@ unless => "/bin/true # comment to satisfy puppet syntax requirements set -ex ceph osd lspools | grep ' ${name},'", + timeout => $exec_timeout, } exec { "set-${name}-pg_num": @@ -72,6 +77,7 @@ set -ex test $(ceph osd pool get ${name} pg_num | sed 's/.*:\s*//g') -ge ${pg_num}", require => Exec["create-${name}"], + timeout => $exec_timeout, } if $pgp_num { @@ -83,6 +89,7 @@ set -ex test $(ceph osd pool get ${name} pgp_num | sed 's/.*:\s*//g') -ge ${pgp_num}", require => Exec["create-${name}"], + timeout => $exec_timeout, } } @@ -95,6 +102,7 @@ set -ex test $(ceph osd pool get ${name} size | sed 's/.*:\s*//g') -eq ${size}", require => Exec["create-${name}"], + timeout => $exec_timeout, } } @@ -107,6 +115,7 @@ onlyif => "/bin/true # comment to satisfy puppet syntax requirements set -ex ceph osd lspools | grep ${name}", + timeout => $exec_timeout, } -> Ceph::Mon<| ensure == absent |> } else { diff --git a/ceph/manifests/profile/base.pp b/ceph/manifests/profile/base.pp index bccf7e121..8a4b648e7 100644 --- a/ceph/manifests/profile/base.pp +++ b/ceph/manifests/profile/base.pp @@ -40,6 +40,7 @@ osd_pool_default_min_size => $ceph::profile::params::osd_pool_default_min_size, mon_initial_members => $ceph::profile::params::mon_initial_members, mon_host => $ceph::profile::params::mon_host, + ms_bind_ipv6 => $ceph::profile::params::ms_bind_ipv6, cluster_network => $ceph::profile::params::cluster_network, public_network => $ceph::profile::params::public_network, } diff --git a/ceph/manifests/profile/params.pp b/ceph/manifests/profile/params.pp index 5f79cdcae..1fcbd93fd 100644 --- a/ceph/manifests/profile/params.pp +++ b/ceph/manifests/profile/params.pp @@ -38,6 +38,9 @@ # individually through ceph::mon. # Optional. String like e.g. 'a, b, c'. # +# [*ms_bind_ipv6*] Enables Ceph daemons to bind to IPv6 addresses. +# Optional. Boolean. Default provided by Ceph. +# # [*osd_journal_size*] The size of the journal file/device. # Optional. Integer. Default provided by Ceph. # @@ -98,6 +101,7 @@ $authentication_type = undef, $mon_initial_members = undef, $mon_host = undef, + $ms_bind_ipv6 = undef, $osd_journal_size = undef, $osd_pool_default_pg_num = undef, $osd_pool_default_pgp_num = undef, diff --git a/ceph/manifests/repo.pp b/ceph/manifests/repo.pp index 5f791efa4..b17c6d093 100644 --- a/ceph/manifests/repo.pp +++ b/ceph/manifests/repo.pp @@ -107,7 +107,11 @@ $el = '7' } - if ($::operatingsystem == 'CentOS') { + # Firefly is the last ceph.com supported release which conflicts with + # the CentOS 7 base channel. Therefore make sure to only exclude the + # conflicting packages in the exact combination of CentOS7 and Firefly. + # TODO: Remove this once Firefly becomes EOL + if ($::operatingsystem == 'CentOS' and $el == '7' and $release == 'firefly') { file_line { 'exclude base': ensure => $ensure, path => '/etc/yum.repos.d/CentOS-Base.repo', diff --git a/ceph/spec/classes/ceph_init_spec.rb b/ceph/spec/classes/ceph_init_spec.rb index a08baf94c..ae2c187cf 100644 --- a/ceph/spec/classes/ceph_init_spec.rb +++ b/ceph/spec/classes/ceph_init_spec.rb @@ -41,12 +41,14 @@ it { is_expected.to_not contain_ceph_config('global/mon_osd_nearfull_ratio').with_value('85') } it { is_expected.to_not contain_ceph_config('global/mon_initial_members').with_value('mon.01') } it { is_expected.to_not contain_ceph_config('global/mon_host').with_value('mon01.ceph, mon02.ceph') } + it { is_expected.to_not contain_ceph_config('global/ms_bind_ipv6').with_value('false') } it { is_expected.to_not contain_ceph_config('global/require_signatures').with_value('false') } it { is_expected.to_not contain_ceph_config('global/cluster_require_signatures').with_value('false') } it { is_expected.to_not contain_ceph_config('global/service_require_signatures').with_value('false') } it { is_expected.to_not contain_ceph_config('global/sign_messages').with_value('true') } it { is_expected.to_not contain_ceph_config('global/cluster_network').with_value('10.0.0.0/24') } it { is_expected.to_not contain_ceph_config('global/public_network').with_value('192.168.0.0/24') } + it { is_expected.to_not contain_ceph_config('global/public_addr').with_value('192.168.0.2') } it { is_expected.to_not contain_ceph_config('osd/osd_journal_size').with_value('4096') } it { is_expected.to contain_ceph_config('global/auth_cluster_required').with_value('cephx') } it { is_expected.to contain_ceph_config('global/auth_service_required').with_value('cephx') } @@ -74,12 +76,14 @@ :mon_osd_nearfull_ratio => '90', :mon_initial_members => 'mon.01', :mon_host => 'mon01.ceph, mon02.ceph', + :ms_bind_ipv6 => 'true', :require_signatures => 'true', :cluster_require_signatures => 'true', :service_require_signatures => 'true', :sign_messages => 'false', :cluster_network => '10.0.0.0/24', :public_network => '192.168.0.0/24', + :public_addr => '192.168.0.2', } end @@ -98,12 +102,14 @@ it { is_expected.to contain_ceph_config('global/mon_osd_nearfull_ratio').with_value('90') } it { is_expected.to contain_ceph_config('global/mon_initial_members').with_value('mon.01') } it { is_expected.to contain_ceph_config('global/mon_host').with_value('mon01.ceph, mon02.ceph') } + it { is_expected.to contain_ceph_config('global/ms_bind_ipv6').with_value('true') } it { is_expected.to contain_ceph_config('global/require_signatures').with_value('true') } it { is_expected.to contain_ceph_config('global/cluster_require_signatures').with_value('true') } it { is_expected.to contain_ceph_config('global/service_require_signatures').with_value('true') } it { is_expected.to contain_ceph_config('global/sign_messages').with_value('false') } it { is_expected.to contain_ceph_config('global/cluster_network').with_value('10.0.0.0/24') } it { is_expected.to contain_ceph_config('global/public_network').with_value('192.168.0.0/24') } + it { is_expected.to contain_ceph_config('global/public_addr').with_value('192.168.0.2') } it { is_expected.to contain_ceph_config('osd/osd_journal_size').with_value('1024') } it { is_expected.to_not contain_ceph_config('global/auth_cluster_required').with_value('cephx') } it { is_expected.to_not contain_ceph_config('global/auth_service_required').with_value('cephx') } diff --git a/ceph/spec/classes/ceph_repo_spec.rb b/ceph/spec/classes/ceph_repo_spec.rb index 4ff9dd820..92eb452ed 100644 --- a/ceph/spec/classes/ceph_repo_spec.rb +++ b/ceph/spec/classes/ceph_repo_spec.rb @@ -195,6 +195,8 @@ describe "with default params" do + it { is_expected.not_to contain_file_line('exclude base') } + it { is_expected.to contain_yumrepo('ext-epel-6').with( :enabled => '1', :descr => 'External EPEL 6', @@ -237,6 +239,8 @@ } end + it { is_expected.not_to contain_file_line('exclude base') } + it { is_expected.to contain_yumrepo('ext-epel-6').with( :enabled => '1', :descr => 'External EPEL 6', @@ -281,6 +285,8 @@ } end + it { is_expected.not_to contain_file_line('exclude base') } + it { is_expected.to contain_yumrepo('ext-epel-6').with( :enabled => '0', :descr => 'External EPEL 6', @@ -346,6 +352,8 @@ } end + it { is_expected.not_to contain_file_line('exclude base') } + it { is_expected.to contain_yumrepo('ext-epel-6').with( :enabled => '1', :descr => 'External EPEL 6', @@ -400,6 +408,8 @@ } end + it { is_expected.not_to contain_file_line('exclude base') } + it { is_expected.to contain_yumrepo('ext-epel-6').with( :enabled => '1', :descr => 'External EPEL 6', @@ -460,6 +470,8 @@ describe "with default params" do + it { is_expected.not_to contain_file_line('exclude base') } + it { is_expected.to contain_yumrepo('ext-epel-7').with( :enabled => '1', :descr => 'External EPEL 7', @@ -502,6 +514,8 @@ } end + it { is_expected.not_to contain_file_line('exclude base') } + it { is_expected.to contain_yumrepo('ext-epel-7').with( :enabled => '1', :descr => 'External EPEL 7', @@ -546,6 +560,8 @@ } end + it { is_expected.not_to contain_file_line('exclude base') } + it { is_expected.to contain_yumrepo('ext-epel-7').with( :enabled => '0', :descr => 'External EPEL 7', @@ -602,6 +618,8 @@ } end + it { is_expected.not_to contain_file_line('exclude base') } + it { is_expected.to contain_yumrepo('ext-epel-7').with( :enabled => '1', :descr => 'External EPEL 7', @@ -646,6 +664,545 @@ } end + it { is_expected.not_to contain_file_line('exclude base') } + + it { is_expected.to contain_yumrepo('ext-epel-7').with( + :enabled => '1', + :descr => 'External EPEL 7', + :name => 'ext-epel-7', + :baseurl => 'absent', + :gpgcheck => '1', + :gpgkey => 'https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7', + :mirrorlist => 'http://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch', + :priority => '20', + :exclude => 'python-ceph-compat python-rbd python-rados python-cephfs', + ) } + + it { is_expected.to contain_yumrepo('ext-ceph').with( + :enabled => '1', + :descr => 'External Ceph hammer', + :name => 'ext-ceph-hammer', + :baseurl => 'http://ceph.com/rpm-hammer/el7/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-noarch').with( + :enabled => '1', + :descr => 'External Ceph noarch', + :name => 'ext-ceph-hammer-noarch', + :baseurl => 'http://ceph.com/rpm-hammer/el7/noarch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-fastcgi').with( + :enabled => '1', + :descr => 'FastCGI basearch packages for Ceph', + :name => 'ext-ceph-fastcgi', + :baseurl => 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel7-x86_64-basic/ref/master', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/autobuild.asc', + :mirrorlist => 'absent', + :priority => '20' + ) } + + end + end + + + describe 'CentOS6' do + + let :facts do + { + :osfamily => 'RedHat', + :operatingsystem => 'CentOS', + :operatingsystemmajrelease => '6', + } + end + + describe "with default params" do + + it { is_expected.not_to contain_file_line('exclude base') } + + it { is_expected.to contain_yumrepo('ext-epel-6').with( + :enabled => '1', + :descr => 'External EPEL 6', + :name => 'ext-epel-6', + :baseurl => 'absent', + :gpgcheck => '1', + :gpgkey => 'https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-6', + :mirrorlist => 'http://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch', + :priority => '20', + :exclude => 'python-ceph-compat python-rbd python-rados python-cephfs', + ) } + + it { is_expected.to contain_yumrepo('ext-ceph').with( + :enabled => '1', + :descr => 'External Ceph hammer', + :name => 'ext-ceph-hammer', + :baseurl => 'http://ceph.com/rpm-hammer/el6/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-noarch').with( + :enabled => '1', + :descr => 'External Ceph noarch', + :name => 'ext-ceph-hammer-noarch', + :baseurl => 'http://ceph.com/rpm-hammer/el6/noarch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + end + + describe "when overriding ceph release" do + let :params do + { + :release => 'firefly' + } + end + + it { is_expected.not_to contain_file_line('exclude base') } + + it { is_expected.to contain_yumrepo('ext-epel-6').with( + :enabled => '1', + :descr => 'External EPEL 6', + :name => 'ext-epel-6', + :baseurl => 'absent', + :gpgcheck => '1', + :gpgkey => 'https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-6', + :mirrorlist => 'http://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch', + :priority => '20', + :exclude => 'python-ceph-compat python-rbd python-rados python-cephfs', + ) } + + it { is_expected.to contain_yumrepo('ext-ceph').with( + :enabled => '1', + :descr => 'External Ceph firefly', + :name => 'ext-ceph-firefly', + :baseurl => 'http://ceph.com/rpm-firefly/el6/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-noarch').with( + :enabled => '1', + :descr => 'External Ceph noarch', + :name => 'ext-ceph-firefly-noarch', + :baseurl => 'http://ceph.com/rpm-firefly/el6/noarch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + end + + describe "with ensure => absent to disable" do + let :params do + { + :ensure => 'absent', + :extras => true, + :fastcgi => true + } + end + + it { is_expected.not_to contain_file_line('exclude base') } + + it { is_expected.to contain_yumrepo('ext-epel-6').with( + :enabled => '0', + :descr => 'External EPEL 6', + :name => 'ext-epel-6', + :baseurl => 'absent', + :gpgcheck => '1', + :gpgkey => 'https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-6', + :mirrorlist => 'http://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch', + :priority => '20', + :exclude => 'python-ceph-compat python-rbd python-rados python-cephfs', + ) } + + it { is_expected.to contain_yumrepo('ext-ceph').with( + :enabled => '0', + :descr => 'External Ceph hammer', + :name => 'ext-ceph-hammer', + :baseurl => 'http://ceph.com/rpm-hammer/el6/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-noarch').with( + :enabled => '0', + :descr => 'External Ceph noarch', + :name => 'ext-ceph-hammer-noarch', + :baseurl => 'http://ceph.com/rpm-hammer/el6/noarch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-extras').with( + :enabled => '0', + :descr => 'External Ceph Extras', + :name => 'ext-ceph-extras', + :baseurl => 'http://ceph.com/packages/ceph-extras/rpm/rhel6/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-fastcgi').with( + :enabled => '0', + :descr => 'FastCGI basearch packages for Ceph', + :name => 'ext-ceph-fastcgi', + :baseurl => 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel6-x86_64-basic/ref/master', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/autobuild.asc', + :mirrorlist => 'absent', + :priority => '20' + ) } + + end + + describe "with ceph extras" do + let :params do + { + :extras => true + } + end + + it { is_expected.not_to contain_file_line('exclude base') } + + it { is_expected.to contain_yumrepo('ext-epel-6').with( + :enabled => '1', + :descr => 'External EPEL 6', + :name => 'ext-epel-6', + :baseurl => 'absent', + :gpgcheck => '1', + :gpgkey => 'https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-6', + :mirrorlist => 'http://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch', + :priority => '20', + :exclude => 'python-ceph-compat python-rbd python-rados python-cephfs', + ) } + + it { is_expected.to contain_yumrepo('ext-ceph').with( + :enabled => '1', + :descr => 'External Ceph hammer', + :name => 'ext-ceph-hammer', + :baseurl => 'http://ceph.com/rpm-hammer/el6/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-noarch').with( + :enabled => '1', + :descr => 'External Ceph noarch', + :name => 'ext-ceph-hammer-noarch', + :baseurl => 'http://ceph.com/rpm-hammer/el6/noarch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-extras').with( + :enabled => '1', + :descr => 'External Ceph Extras', + :name => 'ext-ceph-extras', + :baseurl => 'http://ceph.com/packages/ceph-extras/rpm/rhel6/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + end + + describe "with ceph fast-cgi" do + let :params do + { + :fastcgi => true + } + end + + it { is_expected.not_to contain_file_line('exclude base') } + + it { is_expected.to contain_yumrepo('ext-epel-6').with( + :enabled => '1', + :descr => 'External EPEL 6', + :name => 'ext-epel-6', + :baseurl => 'absent', + :gpgcheck => '1', + :gpgkey => 'https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-6', + :mirrorlist => 'http://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch', + :priority => '20', + :exclude => 'python-ceph-compat python-rbd python-rados python-cephfs', + ) } + + it { is_expected.to contain_yumrepo('ext-ceph').with( + :enabled => '1', + :descr => 'External Ceph hammer', + :name => 'ext-ceph-hammer', + :baseurl => 'http://ceph.com/rpm-hammer/el6/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-noarch').with( + :enabled => '1', + :descr => 'External Ceph noarch', + :name => 'ext-ceph-hammer-noarch', + :baseurl => 'http://ceph.com/rpm-hammer/el6/noarch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-fastcgi').with( + :enabled => '1', + :descr => 'FastCGI basearch packages for Ceph', + :name => 'ext-ceph-fastcgi', + :baseurl => 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel6-x86_64-basic/ref/master', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/autobuild.asc', + :mirrorlist => 'absent', + :priority => '20' + ) } + + end + end + + describe 'CentOS7' do + + let :facts do + { + :osfamily => 'RedHat', + :operatingsystem => 'CentOS', + :operatingsystemmajrelease => '7', + } + end + + describe "with default params" do + + it { is_expected.not_to contain_file_line('exclude base') } + + it { is_expected.to contain_yumrepo('ext-epel-7').with( + :enabled => '1', + :descr => 'External EPEL 7', + :name => 'ext-epel-7', + :baseurl => 'absent', + :gpgcheck => '1', + :gpgkey => 'https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7', + :mirrorlist => 'http://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch', + :priority => '20', + :exclude => 'python-ceph-compat python-rbd python-rados python-cephfs', + ) } + + it { is_expected.to contain_yumrepo('ext-ceph').with( + :enabled => '1', + :descr => 'External Ceph hammer', + :name => 'ext-ceph-hammer', + :baseurl => 'http://ceph.com/rpm-hammer/el7/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-noarch').with( + :enabled => '1', + :descr => 'External Ceph noarch', + :name => 'ext-ceph-hammer-noarch', + :baseurl => 'http://ceph.com/rpm-hammer/el7/noarch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + end + + describe "when overriding ceph release" do + let :params do + { + :release => 'firefly' + } + end + + it { is_expected.to contain_file_line('exclude base').with( + :ensure => 'present', + :path => '/etc/yum.repos.d/CentOS-Base.repo', + :after => '^\[base\]$', + :line => 'exclude=python-ceph-compat python-rbd python-rados python-cephfs', + ) } + + it { is_expected.to contain_yumrepo('ext-epel-7').with( + :enabled => '1', + :descr => 'External EPEL 7', + :name => 'ext-epel-7', + :baseurl => 'absent', + :gpgcheck => '1', + :gpgkey => 'https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7', + :mirrorlist => 'http://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch', + :priority => '20', + :exclude => 'python-ceph-compat python-rbd python-rados python-cephfs', + ) } + + it { is_expected.to contain_yumrepo('ext-ceph').with( + :enabled => '1', + :descr => 'External Ceph firefly', + :name => 'ext-ceph-firefly', + :baseurl => 'http://ceph.com/rpm-firefly/el7/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-noarch').with( + :enabled => '1', + :descr => 'External Ceph noarch', + :name => 'ext-ceph-firefly-noarch', + :baseurl => 'http://ceph.com/rpm-firefly/el7/noarch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + end + + describe "with ensure => absent to disable" do + let :params do + { + :ensure => 'absent', + :extras => true, + :fastcgi => true + } + end + + it { is_expected.not_to contain_file_line('exclude base') } + + it { is_expected.to contain_yumrepo('ext-epel-7').with( + :enabled => '0', + :descr => 'External EPEL 7', + :name => 'ext-epel-7', + :baseurl => 'absent', + :gpgcheck => '1', + :gpgkey => 'https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7', + :mirrorlist => 'http://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch', + :priority => '20', + :exclude => 'python-ceph-compat python-rbd python-rados python-cephfs', + ) } + + it { is_expected.to contain_yumrepo('ext-ceph').with( + :enabled => '0', + :descr => 'External Ceph hammer', + :name => 'ext-ceph-hammer', + :baseurl => 'http://ceph.com/rpm-hammer/el7/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-noarch').with( + :enabled => '0', + :descr => 'External Ceph noarch', + :name => 'ext-ceph-hammer-noarch', + :baseurl => 'http://ceph.com/rpm-hammer/el7/noarch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.not_to contain_yumrepo('ext-ceph-extras') } + + it { is_expected.to contain_yumrepo('ext-ceph-fastcgi').with( + :enabled => '0', + :descr => 'FastCGI basearch packages for Ceph', + :name => 'ext-ceph-fastcgi', + :baseurl => 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-rhel7-x86_64-basic/ref/master', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/autobuild.asc', + :mirrorlist => 'absent', + :priority => '20' + ) } + + end + + describe "with ceph extras" do + let :params do + { + :extras => true + } + end + + it { is_expected.not_to contain_file_line('exclude base') } + + it { is_expected.to contain_yumrepo('ext-epel-7').with( + :enabled => '1', + :descr => 'External EPEL 7', + :name => 'ext-epel-7', + :baseurl => 'absent', + :gpgcheck => '1', + :gpgkey => 'https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7', + :mirrorlist => 'http://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch', + :priority => '20', + :exclude => 'python-ceph-compat python-rbd python-rados python-cephfs', + ) } + + it { is_expected.to contain_yumrepo('ext-ceph').with( + :enabled => '1', + :descr => 'External Ceph hammer', + :name => 'ext-ceph-hammer', + :baseurl => 'http://ceph.com/rpm-hammer/el7/$basearch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.to contain_yumrepo('ext-ceph-noarch').with( + :enabled => '1', + :descr => 'External Ceph noarch', + :name => 'ext-ceph-hammer-noarch', + :baseurl => 'http://ceph.com/rpm-hammer/el7/noarch', + :gpgcheck => '1', + :gpgkey => 'https://git.ceph.com/release.asc', + :mirrorlist => 'absent', + :priority => '10' + ) } + + it { is_expected.not_to contain_yumrepo('ext-ceph-extras') } + end + + describe "with ceph fast-cgi" do + let :params do + { + :fastcgi => true + } + end + + it { is_expected.not_to contain_file_line('exclude base') } + it { is_expected.to contain_yumrepo('ext-epel-7').with( :enabled => '1', :descr => 'External EPEL 7', diff --git a/ceph/spec/defines/ceph_mon_spec.rb b/ceph/spec/defines/ceph_mon_spec.rb index 5c4656f32..ee52c282a 100644 --- a/ceph/spec/defines/ceph_mon_spec.rb +++ b/ceph/spec/defines/ceph_mon_spec.rb @@ -234,7 +234,10 @@ it { is_expected.to contain_exec('ceph-mon-ceph.client.admin.keyring-A').with( 'command' => '/bin/true # comment to satisfy puppet syntax requirements set -ex -touch /etc/ceph/ceph.client.admin.keyring' +touch /etc/ceph/ceph.client.admin.keyring', + 'unless' => '/bin/true # comment to satisfy puppet syntax requirements +set -ex +test -e /etc/ceph/ceph.client.admin.keyring' ) } it { is_expected.to contain_exec('ceph-mon-mkfs-A').with( 'command' => "/bin/true # comment to satisfy puppet syntax requirements @@ -257,7 +260,12 @@ it { is_expected.to contain_file('/tmp/ceph-mon-keyring-A').with( 'mode' => '0444', 'content' => "[mon.]\n\tkey = AQATGHJTUCBqIBAA7M2yafV1xctn1pgr3GcKPg==\n\tcaps mon = \"allow *\"\n") } - it { is_expected.to contain_exec('rm-keyring-A').with('command' => '/bin/rm /tmp/ceph-mon-keyring-A') } + it { is_expected.to contain_exec('rm-keyring-A').with( + 'command' => '/bin/rm /tmp/ceph-mon-keyring-A', + 'unless' => '/bin/true # comment to satisfy puppet syntax requirements +set -ex +test ! -e /tmp/ceph-mon-keyring-A +') } end describe 'with keyring' do From dd131c8ecb81bb5241c74ec6faa9c36cb97708c9 Mon Sep 17 00:00:00 2001 From: Lukas Bezdicka Date: Thu, 28 Jan 2016 17:00:00 +0100 Subject: [PATCH 20/35] Add support for PyMySQL Update ironic to 92ba5d6f7be0f1a24664888f4a8767ab3ab9e703 92ba5d6f7be0f1a24664888f4a8767ab3ab9e703 Merge "Support of PyMySQL driver for MySQL backend" into stable/liberty 884b2c08fe06b88c46b03ac54ea69b2d3f303690 Support of PyMySQL driver for MySQL backend 958ea03e70162f90ffa2b11cabf42990bd39e8d1 Fix duplicate parameters and drop mock Change-Id: I78eff6d5f7f22f0a72a197b3068ec0a76a2aa442 Update sahara to 64a98ee7f6d89c4705762b26ac59796a430b451c 64a98ee7f6d89c4705762b26ac59796a430b451c Support of PyMySQL driver for MySQL backend 82bb6c7daf2bad8e990346bf745367f12d4d4a7b Remove duplicate params and drop mock Change-Id: Ic9735c2db6097adf3be715bae4cf7afdba87a637 --- Puppetfile | 4 +-- ironic/Gemfile | 1 - ironic/manifests/db.pp | 12 +++++-- ironic/manifests/init.pp | 10 ------ ironic/manifests/keystone/auth.pp | 1 - ironic/manifests/params.pp | 2 ++ ironic/spec/acceptance/basic_ironic_spec.rb | 2 +- ironic/spec/classes/ironic_db_spec.rb | 38 ++++++++++++++++++-- ironic/spec/spec_helper.rb | 1 - sahara/Gemfile | 1 - sahara/examples/basic.pp | 2 +- sahara/manifests/db.pp | 17 ++++++--- sahara/manifests/init.pp | 1 - sahara/manifests/params.pp | 2 ++ sahara/spec/acceptance/basic_sahara_spec.rb | 2 +- sahara/spec/classes/sahara_db_spec.rb | 39 +++++++++++++++++++-- sahara/spec/spec_helper.rb | 1 - 17 files changed, 102 insertions(+), 34 deletions(-) diff --git a/Puppetfile b/Puppetfile index c89537e29..7c1c39d2b 100644 --- a/Puppetfile +++ b/Puppetfile @@ -103,7 +103,7 @@ mod 'ipa', :git => 'https://github.com/xbezdick/puppet-ipa-1.git' mod 'ironic', - :commit => 'd97c722fa3c147b9ab45f4837dc5ced4ea642d6b', + :commit => '92ba5d6f7be0f1a24664888f4a8767ab3ab9e703', :git => 'https://github.com/openstack/puppet-ironic.git' mod 'java', @@ -219,7 +219,7 @@ mod 'rsync', :git => 'https://github.com/puppetlabs/puppetlabs-rsync.git' mod 'sahara', - :commit => '64774e9cf344f897163b49afcb725ccc62689610', + :commit => '64a98ee7f6d89c4705762b26ac59796a430b451c', :git => 'https://github.com/openstack/puppet-sahara.git' mod 'sensu', diff --git a/ironic/Gemfile b/ironic/Gemfile index fc2214398..984392186 100644 --- a/ironic/Gemfile +++ b/ironic/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/ironic/manifests/db.pp b/ironic/manifests/db.pp index 92c805f2e..425adad73 100644 --- a/ironic/manifests/db.pp +++ b/ironic/manifests/db.pp @@ -43,6 +43,8 @@ $database_max_overflow = 20, ) { + include ::ironic::params + # NOTE(spredzy): In order to keep backward compatibility we rely on the pick function # to use ironic:: if ironic::db:: isn't specified. $database_connection_real = pick($::ironic::database_connection, $database_connection) @@ -54,14 +56,18 @@ $database_max_overflow_real = pick($::ironic::database_max_overflow, $database_max_overflow) validate_re($database_connection_real, - '(sqlite|mysql|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') + '^(sqlite|mysql(\+pymysql)?|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') if $database_connection_real { case $database_connection_real { - /^mysql:\/\//: { - $backend_package = false + /^mysql(\+pymysql)?:\/\//: { require 'mysql::bindings' require 'mysql::bindings::python' + if $database_connection_real =~ /^mysql\+pymysql/ { + $backend_package = $::ironic::params::pymysql_package_name + } else { + $backend_package = false + } } /^postgresql:\/\//: { $backend_package = false diff --git a/ironic/manifests/init.pp b/ironic/manifests/init.pp index 0bd03662c..c52f779d1 100644 --- a/ironic/manifests/init.pp +++ b/ironic/manifests/init.pp @@ -113,12 +113,6 @@ # [*rabbit_virtual_host*] # (optional) Various rabbitmq settings # -# [*rabbit_hosts*] -# (optional) array of rabbitmq servers for HA. -# A single IP address, such as a VIP, can be used for load-balancing -# multiple RabbitMQ Brokers. -# Defaults to false -# # [*qpid_hostname*] # [*qpid_port*] # [*qpid_username*] @@ -217,8 +211,6 @@ $rabbit_virtual_host = '/', $rabbit_host = 'localhost', $rabbit_port = 5672, - $rabbit_hosts = false, - $rabbit_virtual_host = '/', $rabbit_userid = 'guest', $rabbit_password = false, $rabbit_use_ssl = false, @@ -240,8 +232,6 @@ $qpid_reconnect_interval_min = 0, $qpid_reconnect_interval_max = 0, $qpid_reconnect_interval = 0, - $use_syslog = false, - $log_facility = 'LOG_USER', $database_connection = undef, $database_max_retries = undef, $database_idle_timeout = undef, diff --git a/ironic/manifests/keystone/auth.pp b/ironic/manifests/keystone/auth.pp index 63d944fa0..f258667c3 100644 --- a/ironic/manifests/keystone/auth.pp +++ b/ironic/manifests/keystone/auth.pp @@ -124,7 +124,6 @@ $service_name = undef, $service_type = 'baremetal', $service_description = 'Ironic Bare Metal Provisioning Service', - $public_protocol = 'http', $region = 'RegionOne', $public_url = 'http://127.0.0.1:6385', $admin_url = 'http://127.0.0.1:6385', diff --git a/ironic/manifests/params.pp b/ironic/manifests/params.pp index 700ac9e11..dc9f3c396 100644 --- a/ironic/manifests/params.pp +++ b/ironic/manifests/params.pp @@ -38,6 +38,7 @@ $inspector_service = 'openstack-ironic-inspector' $inspector_dnsmasq_service = 'openstack-ironic-inspector-dnsmasq' $sqlite_package_name = undef + $pymysql_package_name = undef } 'Debian': { $common_package_name = 'ironic-common' @@ -53,6 +54,7 @@ # this should be changed to whatever debian will use for dnsmasq $inspector_dnsmasq_service = 'ironic-inspector-dnsmasq' $sqlite_package_name = 'python-pysqlite2' + $pymysql_package_name = 'python-pymysql' } default: { fail("Unsupported osfamily ${::osfamily}") diff --git a/ironic/spec/acceptance/basic_ironic_spec.rb b/ironic/spec/acceptance/basic_ironic_spec.rb index dd8ed08c1..26c8ed487 100644 --- a/ironic/spec/acceptance/basic_ironic_spec.rb +++ b/ironic/spec/acceptance/basic_ironic_spec.rb @@ -32,7 +32,7 @@ class { '::ironic': rabbit_userid => 'ironic', rabbit_password => 'an_even_bigger_secret', rabbit_host => '127.0.0.1', - database_connection => 'mysql://ironic:a_big_secret@127.0.0.1/ironic?charset=utf8', + database_connection => 'mysql+pymysql://ironic:a_big_secret@127.0.0.1/ironic?charset=utf8', debug => true, verbose => true, enabled_drivers => ['pxe_ssh'], diff --git a/ironic/spec/classes/ironic_db_spec.rb b/ironic/spec/classes/ironic_db_spec.rb index e152a7a26..5e129b32d 100644 --- a/ironic/spec/classes/ironic_db_spec.rb +++ b/ironic/spec/classes/ironic_db_spec.rb @@ -18,7 +18,7 @@ context 'with specific parameters' do let :params do - { :database_connection => 'mysql://ironic:ironic@localhost/ironic', + { :database_connection => 'mysql+pymysql://ironic:ironic@localhost/ironic', :database_idle_timeout => '3601', :database_min_pool_size => '2', :database_max_pool_size => '21', @@ -27,7 +27,7 @@ :database_retry_interval => '11', } end - it { is_expected.to contain_ironic_config('database/connection').with_value('mysql://ironic:ironic@localhost/ironic').with_secret(true) } + it { is_expected.to contain_ironic_config('database/connection').with_value('mysql+pymysql://ironic:ironic@localhost/ironic').with_secret(true)} it { is_expected.to contain_ironic_config('database/idle_timeout').with_value('3601') } it { is_expected.to contain_ironic_config('database/min_pool_size').with_value('2') } it { is_expected.to contain_ironic_config('database/max_retries').with_value('11') } @@ -37,6 +37,16 @@ end + + context 'with MySQL-python library as backend package' do + let :params do + { :database_connection => 'mysql://ironic:ironic@localhost/ironic' } + end + + it { is_expected.to contain_ironic_config('database/connection').with_value('mysql://ironic:ironic@localhost/ironic').with_secret(true) } + end + + context 'with postgresql backend' do let :params do { :database_connection => 'postgresql://ironic:ironic@localhost/ironic', } @@ -56,8 +66,16 @@ it_raises 'a Puppet::Error', /validate_re/ end + context 'with incorrect database_connection string' do + let :params do + { :database_connection => 'foo+pymysql://ironic:ironic@localhost/ironic', } + end + + it_raises 'a Puppet::Error', /validate_re/ + end end + context 'on Debian platforms' do let :facts do { :osfamily => 'Debian', @@ -67,6 +85,14 @@ end it_configures 'ironic::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://ironic:ironic@localhost/ironic' } + end + + it { is_expected.to contain_package('ironic-backend-package').with({ :ensure => 'present', :name => 'python-pymysql' }) } + end end context 'on Redhat platforms' do @@ -77,6 +103,14 @@ end it_configures 'ironic::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://ironic:ironic@localhost/ironic' } + end + + it { is_expected.not_to contain_package('ironic-backend-package') } + end end end diff --git a/ironic/spec/spec_helper.rb b/ironic/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/ironic/spec/spec_helper.rb +++ b/ironic/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/sahara/Gemfile b/sahara/Gemfile index fc2214398..984392186 100644 --- a/sahara/Gemfile +++ b/sahara/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/sahara/examples/basic.pp b/sahara/examples/basic.pp index ded521d7c..9fa262300 100644 --- a/sahara/examples/basic.pp +++ b/sahara/examples/basic.pp @@ -22,7 +22,7 @@ # Then the common class class { '::sahara': - database_connection => 'mysql://sahara:a_big_secret@127.0.0.1:3306/sahara', + database_connection => 'mysql+pymysql://sahara:a_big_secret@127.0.0.1:3306/sahara', verbose => true, debug => true, admin_user => 'admin', diff --git a/sahara/manifests/db.pp b/sahara/manifests/db.pp index 32925b8e0..6affc89f1 100644 --- a/sahara/manifests/db.pp +++ b/sahara/manifests/db.pp @@ -6,7 +6,7 @@ # # [*database_connection*] # (Optional) The connection string to use to connect to the database. -# Defaults to 'mysql://sahara:secrete@localhost:3306/sahara' +# Defaults to 'mysql+pymysql://sahara:secrete@localhost:3306/sahara' # # [*database_max_retries*] # (Optional) Maximum number of database connection retries during startup. @@ -34,7 +34,7 @@ # Defaults to 20. # class sahara::db ( - $database_connection = 'mysql://sahara:secrete@localhost:3306/sahara', + $database_connection = 'mysql+pymysql://sahara:secrete@localhost:3306/sahara', $database_idle_timeout = 3600, $database_min_pool_size = 1, $database_max_pool_size = 10, @@ -43,6 +43,8 @@ $database_max_overflow = 20, ) { + include ::sahara::params + # NOTE(degorenko): In order to keep backward compatibility we rely on the pick function # to use sahara:: if sahara::db:: isn't specified. $database_connection_real = pick($::sahara::database_connection, $database_connection) @@ -53,13 +55,18 @@ $database_retry_interval_real = pick($::sahara::database_retry_interval, $database_retry_interval) $database_max_overflow_real = pick($::sahara::database_max_overflow, $database_max_overflow) - validate_re($database_connection_real, '(mysql|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') + validate_re($database_connection_real, + '^(mysql(\+pymysql)?|postgresql):\/\/(\S+:\S+@\S+\/\S+)?') case $database_connection_real { - /^mysql:\/\//: { - $backend_package = false + /^mysql(\+pymysql)?:\/\//: { require mysql::bindings require mysql::bindings::python + if $database_connection_real =~ /^mysql\+pymysql/ { + $backend_package = $::sahara::params::pymysql_package_name + } else { + $backend_package = false + } } /^postgresql:\/\//: { $backend_package = false diff --git a/sahara/manifests/init.pp b/sahara/manifests/init.pp index d226704c2..f76d0b24e 100644 --- a/sahara/manifests/init.pp +++ b/sahara/manifests/init.pp @@ -313,7 +313,6 @@ $cert_file = undef, $key_file = undef, $database_connection = undef, - $database_max_retries = undef, $database_idle_timeout = undef, $database_min_pool_size = undef, $database_max_pool_size = undef, diff --git a/sahara/manifests/params.pp b/sahara/manifests/params.pp index e830b0db9..47671236a 100644 --- a/sahara/manifests/params.pp +++ b/sahara/manifests/params.pp @@ -15,6 +15,7 @@ $all_service_name = 'openstack-sahara-all' $api_service_name = 'openstack-sahara-api' $engine_service_name = 'openstack-sahara-engine' + $pymysql_package_name = undef } 'Debian': { $common_package_name = 'sahara-common' @@ -24,6 +25,7 @@ $all_service_name = 'sahara' $api_service_name = 'sahara-api' $engine_service_name = 'sahara-engine' + $pymysql_package_name = 'python-pymysql' } default: { fail("Unsupported osfamily: ${::osfamily} operatingsystem: ${::operatingsystem}") diff --git a/sahara/spec/acceptance/basic_sahara_spec.rb b/sahara/spec/acceptance/basic_sahara_spec.rb index f39e3cf97..000060239 100644 --- a/sahara/spec/acceptance/basic_sahara_spec.rb +++ b/sahara/spec/acceptance/basic_sahara_spec.rb @@ -36,7 +36,7 @@ class { '::sahara': rabbit_password => 'an_even_bigger_secret', rabbit_host => '127.0.0.1', rpc_backend => 'rabbit', - database_connection => 'mysql://sahara:a_big_secret@127.0.0.1/sahara?charset=utf8', + database_connection => 'mysql+pymysql://sahara:a_big_secret@127.0.0.1/sahara?charset=utf8', admin_password => 'a_big_secret', } class { '::sahara::service::api': } diff --git a/sahara/spec/classes/sahara_db_spec.rb b/sahara/spec/classes/sahara_db_spec.rb index c39f3b6c9..f9bd84f41 100644 --- a/sahara/spec/classes/sahara_db_spec.rb +++ b/sahara/spec/classes/sahara_db_spec.rb @@ -4,7 +4,7 @@ shared_examples 'sahara::db' do context 'with default parameters' do - it { is_expected.to contain_sahara_config('database/connection').with_value('mysql://sahara:secrete@localhost:3306/sahara').with_secret(true) } + it { is_expected.to contain_sahara_config('database/connection').with_value('mysql+pymysql://sahara:secrete@localhost:3306/sahara').with_secret(true) } it { is_expected.to contain_sahara_config('database/idle_timeout').with_value('3600') } it { is_expected.to contain_sahara_config('database/min_pool_size').with_value('1') } it { is_expected.to contain_sahara_config('database/max_retries').with_value('10') } @@ -15,7 +15,7 @@ context 'with specific parameters' do let :params do - { :database_connection => 'mysql://sahara:sahara@localhost/sahara', + { :database_connection => 'mysql+pymysql://sahara:sahara@localhost/sahara', :database_idle_timeout => '3601', :database_min_pool_size => '2', :database_max_retries => '11', @@ -25,7 +25,7 @@ } end - it { is_expected.to contain_sahara_config('database/connection').with_value('mysql://sahara:sahara@localhost/sahara').with_secret(true) } + it { is_expected.to contain_sahara_config('database/connection').with_value('mysql+pymysql://sahara:sahara@localhost/sahara').with_secret(true) } it { is_expected.to contain_sahara_config('database/idle_timeout').with_value('3601') } it { is_expected.to contain_sahara_config('database/min_pool_size').with_value('2') } it { is_expected.to contain_sahara_config('database/max_retries').with_value('11') } @@ -34,6 +34,14 @@ it { is_expected.to contain_sahara_config('database/max_overflow').with_value('21') } end + context 'with MySQL-python library as backend package' do + let :params do + { :database_connection => 'mysql://sahara:sahara@localhost/sahara' } + end + + it { is_expected.to contain_sahara_config('database/connection').with_value('mysql://sahara:sahara@localhost/sahara').with_secret(true) } + end + context 'with postgresql backend' do let :params do { :database_connection => 'postgresql://sahara:sahara@localhost/sahara', } @@ -52,6 +60,14 @@ it_raises 'a Puppet::Error', /validate_re/ end + + context 'with incorrect database_connection string' do + let :params do + { :database_connection => 'foo+pymysql://sahara:sahara@localhost/sahara', } + end + + it_raises 'a Puppet::Error', /validate_re/ + end end context 'on Debian platforms' do @@ -63,6 +79,15 @@ end it_configures 'sahara::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://sahara:sahara@localhost/sahara' } + end + + it { is_expected.to contain_package('sahara-backend-package').with({ :ensure => 'present', :name => 'python-pymysql' }) } + end + end context 'on Redhat platforms' do @@ -73,6 +98,14 @@ end it_configures 'sahara::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://sahara:sahara@localhost/sahara' } + end + + it { is_expected.not_to contain_package('sahara-backend-package') } + end end end diff --git a/sahara/spec/spec_helper.rb b/sahara/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/sahara/spec/spec_helper.rb +++ b/sahara/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' From 6b76118f0bb62ac93e64360a1aa6cdbaae4b8d97 Mon Sep 17 00:00:00 2001 From: Gael Chamoulaud Date: Wed, 3 Feb 2016 13:52:43 +0100 Subject: [PATCH 21/35] Drop puppet_aviator No longer used since Juno. Change-Id: Ie997f166c03e5c394307c45f07fa98836b14a2ee Signed-off-by: Gael Chamoulaud (cherry picked from commit 0598a2b089d7f3419a6724b3577887f33bf4d22d) --- Puppetfile | 4 - aviator/.gitignore | 1 - aviator/Aviator_README.md | 11 - aviator/Gemfile | 17 - aviator/Gemfile.lock | 81 --- aviator/LICENSE.txt | 22 - aviator/Modulefile | 11 - aviator/README | 45 -- aviator/lib/puppet/feature/aviator.rb | 7 - aviator/lib/puppet/feature/aviator/core.rb | 15 - .../lib/puppet/feature/aviator/core/cli.rb | 2 - .../feature/aviator/core/cli/describer.rb | 161 ------ .../lib/puppet/feature/aviator/core/logger.rb | 55 -- .../puppet/feature/aviator/core/request.rb | 229 -------- .../feature/aviator/core/request_builder.rb | 102 ---- .../puppet/feature/aviator/core/response.rb | 52 -- .../puppet/feature/aviator/core/service.rb | 169 ------ .../puppet/feature/aviator/core/session.rb | 491 ------------------ .../aviator/core/utils/compatibility.rb | 34 -- .../feature/aviator/core/utils/hashish.rb | 127 ----- .../feature/aviator/core/utils/string.rb | 24 - .../common/requests/v0/public/base.rb | 125 ----- .../common/requests/v2/admin/base.rb | 9 - .../common/requests/v2/public/base.rb | 9 - .../common/requests/v3/public/base.rb | 9 - .../v2/admin/confirm_server_resize.rb | 36 -- .../requests/v2/admin/create_network.rb | 56 -- .../requests/v2/admin/get_host_details.rb | 29 -- .../compute/requests/v2/admin/list_hosts.rb | 44 -- .../requests/v2/admin/list_hypervisors.rb | 27 - .../compute/requests/v2/admin/lock_server.rb | 34 -- .../requests/v2/admin/migrate_server.rb | 34 -- .../compute/requests/v2/admin/reset_server.rb | 39 -- .../requests/v2/admin/resize_server.rb | 41 -- .../requests/v2/admin/revert_server_resize.rb | 46 -- .../requests/v2/admin/unlock_server.rb | 34 -- .../requests/v2/public/add_floating_ip.rb | 38 -- .../v2/public/allocate_floating_ip.rb | 35 -- .../v2/public/change_admin_password.rb | 44 -- .../requests/v2/public/create_image.rb | 46 -- .../requests/v2/public/create_keypair.rb | 39 -- .../requests/v2/public/create_server.rb | 55 -- .../requests/v2/public/delete_image.rb | 29 -- .../v2/public/delete_image_metadata_item.rb | 31 -- .../requests/v2/public/delete_server.rb | 29 -- .../v2/public/delete_server_metadata_item.rb | 31 -- .../requests/v2/public/get_flavor_details.rb | 29 -- .../requests/v2/public/get_image_details.rb | 29 -- .../v2/public/get_image_metadata_item.rb | 31 -- .../requests/v2/public/get_network_details.rb | 30 -- .../compute/requests/v2/public/get_server.rb | 28 - .../v2/public/get_server_metadata_item.rb | 31 -- .../requests/v2/public/list_addresses.rb | 36 -- .../requests/v2/public/list_flavors.rb | 35 -- .../requests/v2/public/list_floating_ips.rb | 26 - .../requests/v2/public/list_image_metadata.rb | 30 -- .../compute/requests/v2/public/list_images.rb | 38 -- .../requests/v2/public/list_keypairs.rb | 26 - .../requests/v2/public/list_networks.rb | 27 - .../v2/public/list_server_metadata.rb | 30 -- .../requests/v2/public/list_servers.rb | 58 --- .../requests/v2/public/pause_server.rb | 34 -- .../requests/v2/public/reboot_server.rb | 41 -- .../requests/v2/public/rebuild_server.rb | 53 -- .../requests/v2/public/resume_server.rb | 34 -- .../compute/requests/v2/public/root.rb | 24 - .../requests/v2/public/set_image_metadata.rb | 38 -- .../requests/v2/public/set_server_metadata.rb | 38 -- .../requests/v2/public/suspend_server.rb | 34 -- .../requests/v2/public/unpause_server.rb | 34 -- .../v2/public/update_image_metadata.rb | 38 -- .../requests/v2/public/update_server.rb | 45 -- .../v2/public/update_server_metadata.rb | 38 -- .../v2/admin/add_role_to_user_on_tenant.rb | 33 -- .../requests/v2/admin/create_tenant.rb | 43 -- .../identity/requests/v2/admin/create_user.rb | 67 --- .../admin/delete_role_from_user_on_tenant.rb | 34 -- .../requests/v2/admin/delete_tenant.rb | 29 -- .../identity/requests/v2/admin/delete_user.rb | 39 -- .../requests/v2/admin/get_tenant_by_id.rb | 30 -- .../identity/requests/v2/admin/get_user.rb | 33 -- .../requests/v2/admin/list_tenants.rb | 35 -- .../identity/requests/v2/admin/list_users.rb | 37 -- .../requests/v2/admin/update_tenant.rb | 47 -- .../identity/requests/v2/admin/update_user.rb | 62 --- .../requests/v2/public/create_token.rb | 62 --- .../requests/v2/public/list_tenants.rb | 35 -- .../identity/requests/v2/public/root.rb | 25 - .../requests/v3/public/create_token.rb | 125 ----- .../requests/v1/public/list_public_images.rb | 45 -- .../image/requests/v1/public/root.rb | 25 - .../requests/v1/admin/list_projects.rb | 27 - .../feature/aviator/openstack/provider.rb | 187 ------- .../requests/v1/public/create_volume.rb | 47 -- .../requests/v1/public/delete_volume.rb | 25 - .../volume/requests/v1/public/get_volume.rb | 28 - .../requests/v1/public/list_volume_types.rb | 29 -- .../volume/requests/v1/public/list_volumes.rb | 48 -- .../volume/requests/v1/public/root.rb | 26 - .../requests/v1/public/update_volume.rb | 43 -- aviator/lib/puppet/feature/aviator/version.rb | 3 - aviator/lib/puppet/feature/composite_io.rb | 108 ---- aviator/lib/puppet/feature/faraday.rb | 268 ---------- aviator/lib/puppet/feature/faraday/adapter.rb | 46 -- .../puppet/feature/faraday/adapter/em_http.rb | 237 --------- .../faraday/adapter/em_http_ssl_patch.rb | 56 -- .../feature/faraday/adapter/em_synchrony.rb | 92 ---- .../adapter/em_synchrony/parallel_manager.rb | 66 --- .../puppet/feature/faraday/adapter/excon.rb | 80 --- .../feature/faraday/adapter/httpclient.rb | 106 ---- .../feature/faraday/adapter/net_http.rb | 124 ----- .../faraday/adapter/net_http_persistent.rb | 47 -- .../puppet/feature/faraday/adapter/patron.rb | 72 --- .../puppet/feature/faraday/adapter/rack.rb | 58 --- .../puppet/feature/faraday/adapter/test.rb | 162 ------ .../feature/faraday/adapter/typhoeus.rb | 123 ----- .../lib/puppet/feature/faraday/autoload.rb | 85 --- .../lib/puppet/feature/faraday/connection.rb | 432 --------------- aviator/lib/puppet/feature/faraday/error.rb | 53 -- .../lib/puppet/feature/faraday/middleware.rb | 37 -- aviator/lib/puppet/feature/faraday/options.rb | 350 ------------- .../lib/puppet/feature/faraday/parameters.rb | 193 ------- .../puppet/feature/faraday/rack_builder.rb | 212 -------- aviator/lib/puppet/feature/faraday/request.rb | 92 ---- .../feature/faraday/request/authorization.rb | 42 -- .../faraday/request/basic_authentication.rb | 13 - .../faraday/request/instrumentation.rb | 36 -- .../feature/faraday/request/multipart.rb | 63 --- .../puppet/feature/faraday/request/retry.rb | 140 ----- .../faraday/request/token_authentication.rb | 15 - .../feature/faraday/request/url_encoded.rb | 36 -- .../lib/puppet/feature/faraday/response.rb | 93 ---- .../puppet/feature/faraday/response/logger.rb | 34 -- .../feature/faraday/response/raise_error.rb | 21 - .../lib/puppet/feature/faraday/upload_io.rb | 67 --- aviator/lib/puppet/feature/faraday/utils.rb | 297 ----------- aviator/lib/puppet/feature/multipart_post.rb | 9 - aviator/lib/puppet/feature/multipartable.rb | 29 -- .../puppet/feature/net/http/post/multipart.rb | 27 - aviator/lib/puppet/feature/parts.rb | 96 ---- 140 files changed, 8930 deletions(-) delete mode 100644 aviator/.gitignore delete mode 100644 aviator/Aviator_README.md delete mode 100644 aviator/Gemfile delete mode 100644 aviator/Gemfile.lock delete mode 100644 aviator/LICENSE.txt delete mode 100644 aviator/Modulefile delete mode 100644 aviator/README delete mode 100644 aviator/lib/puppet/feature/aviator.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core/cli.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core/cli/describer.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core/logger.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core/request.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core/request_builder.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core/response.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core/service.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core/session.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core/utils/compatibility.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core/utils/hashish.rb delete mode 100644 aviator/lib/puppet/feature/aviator/core/utils/string.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/common/requests/v0/public/base.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/common/requests/v2/admin/base.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/common/requests/v2/public/base.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/common/requests/v3/public/base.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/confirm_server_resize.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/create_network.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/get_host_details.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/list_hosts.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/list_hypervisors.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/lock_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/migrate_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/reset_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/resize_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/revert_server_resize.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/unlock_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/add_floating_ip.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/allocate_floating_ip.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/change_admin_password.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_image.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_keypair.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_image.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_image_metadata_item.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_server_metadata_item.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_flavor_details.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_image_details.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_image_metadata_item.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_network_details.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_server_metadata_item.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_addresses.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_flavors.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_floating_ips.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_image_metadata.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_images.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_keypairs.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_networks.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_server_metadata.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_servers.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/pause_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/reboot_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/rebuild_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/resume_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/root.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/set_image_metadata.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/set_server_metadata.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/suspend_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/unpause_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_image_metadata.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_server.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_server_metadata.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/add_role_to_user_on_tenant.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/create_tenant.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/create_user.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_role_from_user_on_tenant.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_tenant.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_user.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/get_tenant_by_id.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/get_user.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/list_tenants.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/list_users.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/update_tenant.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/update_user.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/create_token.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/list_tenants.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/root.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/identity/requests/v3/public/create_token.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/image/requests/v1/public/list_public_images.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/image/requests/v1/public/root.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/metering/requests/v1/admin/list_projects.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/provider.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/create_volume.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/delete_volume.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/get_volume.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/list_volume_types.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/list_volumes.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/root.rb delete mode 100644 aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/update_volume.rb delete mode 100644 aviator/lib/puppet/feature/aviator/version.rb delete mode 100644 aviator/lib/puppet/feature/composite_io.rb delete mode 100644 aviator/lib/puppet/feature/faraday.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/em_http.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/em_http_ssl_patch.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/em_synchrony.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/em_synchrony/parallel_manager.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/excon.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/httpclient.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/net_http.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/net_http_persistent.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/patron.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/rack.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/test.rb delete mode 100644 aviator/lib/puppet/feature/faraday/adapter/typhoeus.rb delete mode 100644 aviator/lib/puppet/feature/faraday/autoload.rb delete mode 100644 aviator/lib/puppet/feature/faraday/connection.rb delete mode 100644 aviator/lib/puppet/feature/faraday/error.rb delete mode 100644 aviator/lib/puppet/feature/faraday/middleware.rb delete mode 100644 aviator/lib/puppet/feature/faraday/options.rb delete mode 100644 aviator/lib/puppet/feature/faraday/parameters.rb delete mode 100644 aviator/lib/puppet/feature/faraday/rack_builder.rb delete mode 100644 aviator/lib/puppet/feature/faraday/request.rb delete mode 100644 aviator/lib/puppet/feature/faraday/request/authorization.rb delete mode 100644 aviator/lib/puppet/feature/faraday/request/basic_authentication.rb delete mode 100644 aviator/lib/puppet/feature/faraday/request/instrumentation.rb delete mode 100644 aviator/lib/puppet/feature/faraday/request/multipart.rb delete mode 100644 aviator/lib/puppet/feature/faraday/request/retry.rb delete mode 100644 aviator/lib/puppet/feature/faraday/request/token_authentication.rb delete mode 100644 aviator/lib/puppet/feature/faraday/request/url_encoded.rb delete mode 100644 aviator/lib/puppet/feature/faraday/response.rb delete mode 100644 aviator/lib/puppet/feature/faraday/response/logger.rb delete mode 100644 aviator/lib/puppet/feature/faraday/response/raise_error.rb delete mode 100644 aviator/lib/puppet/feature/faraday/upload_io.rb delete mode 100644 aviator/lib/puppet/feature/faraday/utils.rb delete mode 100644 aviator/lib/puppet/feature/multipart_post.rb delete mode 100644 aviator/lib/puppet/feature/multipartable.rb delete mode 100644 aviator/lib/puppet/feature/net/http/post/multipart.rb delete mode 100644 aviator/lib/puppet/feature/parts.rb diff --git a/Puppetfile b/Puppetfile index 7c1c39d2b..39eb6f6ab 100644 --- a/Puppetfile +++ b/Puppetfile @@ -6,10 +6,6 @@ mod 'apache', :commit => 'cdd81dbf3463c73e6abfaf937f929821baff73dd', :git => 'https://github.com/puppetlabs/puppetlabs-apache.git' -mod 'aviator', - :commit => '9a3b74b0dbb655a3afcbc8bdee442f2143420b72', - :git => 'https://github.com/aimonb/puppet_aviator.git' - mod 'cassandra', :commit => 'b7526c4a8570696396d5f0151fdb9b3d1cbda598', :git => 'https://github.com/locp/cassandra.git' diff --git a/aviator/.gitignore b/aviator/.gitignore deleted file mode 100644 index 5fff1d9c1..000000000 --- a/aviator/.gitignore +++ /dev/null @@ -1 +0,0 @@ -pkg diff --git a/aviator/Aviator_README.md b/aviator/Aviator_README.md deleted file mode 100644 index d85f93a0a..000000000 --- a/aviator/Aviator_README.md +++ /dev/null @@ -1,11 +0,0 @@ -![Aviator](https://raw.github.com/aviator/www/gh-pages/images/logo-small.png) -
A lightweight library for communicating with the OpenStack API. - -[![Build Status](https://travis-ci.org/aviator/aviator.png?branch=master)](https://travis-ci.org/aviator/aviator) -[![Coverage Status](https://coveralls.io/repos/aviator/aviator/badge.png?branch=master)](https://coveralls.io/r/aviator/aviator?branch=master) -[![Code Climate](https://codeclimate.com/github/aviator/aviator.png)](https://codeclimate.com/github/aviator/aviator) -[![Gem Version](https://badge.fury.io/rb/aviator.png)](http://badge.fury.io/rb/aviator) -[![Dependency Status](https://gemnasium.com/aviator/aviator.png)](https://gemnasium.com/aviator/aviator) - - -Usage and Installation diff --git a/aviator/Gemfile b/aviator/Gemfile deleted file mode 100644 index 483cfd0aa..000000000 --- a/aviator/Gemfile +++ /dev/null @@ -1,17 +0,0 @@ -# A sample Gemfile -source "https://rubygems.org" - -gem 'ci_reporter' -gem 'simplecov' -gem 'simplecov-rcov' -gem "rspec", '~> 2.0' -gem "mocha" -gem 'puppet', '= 3.5.1' -gem 'puppet-lint' -gem 'facter', '>= 1.6.10' -gem 'rspec-puppet', :git => 'https://github.com/rodjek/rspec-puppet.git', :branch => 'master' -gem 'rake', '>= 0.9.2' -gem 'puppetlabs_spec_helper', '0.3.0' -gem 'test-unit' -gem 'rspec_junit_formatter' -gem 'rspec-puppet-utils' diff --git a/aviator/Gemfile.lock b/aviator/Gemfile.lock deleted file mode 100644 index b45c0880d..000000000 --- a/aviator/Gemfile.lock +++ /dev/null @@ -1,81 +0,0 @@ -GIT - remote: https://github.com/rodjek/rspec-puppet.git - revision: 389f99ef666521fec1b4530fe69dc1ab84a060a8 - branch: master - specs: - rspec-puppet (1.0.1) - rspec - -GEM - remote: https://rubygems.org/ - specs: - CFPropertyList (2.2.8) - builder (3.2.2) - ci_reporter (2.0.0) - builder (>= 2.1.2) - diff-lcs (1.2.5) - docile (1.1.5) - facter (2.2.0) - CFPropertyList (~> 2.2.6) - hiera (1.3.4) - json_pure - json_pure (1.8.1) - metaclass (0.0.4) - mocha (1.1.0) - metaclass (~> 0.0.1) - multi_json (1.10.1) - power_assert (0.1.3) - puppet (3.5.1) - facter (> 1.6, < 3) - hiera (~> 1.0) - json_pure - rgen (~> 0.6.5) - puppet-lint (1.0.1) - puppetlabs_spec_helper (0.3.0) - mocha (>= 0.10.5) - rake - rspec (>= 2.9.0) - rspec-puppet (>= 0.1.1) - rake (10.3.2) - rgen (0.6.6) - rspec (2.99.0) - rspec-core (~> 2.99.0) - rspec-expectations (~> 2.99.0) - rspec-mocks (~> 2.99.0) - rspec-core (2.99.2) - rspec-expectations (2.99.2) - diff-lcs (>= 1.1.3, < 2.0) - rspec-mocks (2.99.2) - rspec-puppet-utils (2.0.4) - rspec_junit_formatter (0.2.0) - builder (< 4) - rspec (>= 2, < 4) - rspec-core (!= 2.12.0) - simplecov (0.9.0) - docile (~> 1.1.0) - multi_json - simplecov-html (~> 0.8.0) - simplecov-html (0.8.0) - simplecov-rcov (0.2.3) - simplecov (>= 0.4.1) - test-unit (3.0.1) - power_assert - -PLATFORMS - ruby - -DEPENDENCIES - ci_reporter - facter (>= 1.6.10) - mocha - puppet (= 3.5.1) - puppet-lint - puppetlabs_spec_helper (= 0.3.0) - rake (>= 0.9.2) - rspec (~> 2.0) - rspec-puppet! - rspec-puppet-utils - rspec_junit_formatter - simplecov - simplecov-rcov - test-unit diff --git a/aviator/LICENSE.txt b/aviator/LICENSE.txt deleted file mode 100644 index 5c8a6e3a1..000000000 --- a/aviator/LICENSE.txt +++ /dev/null @@ -1,22 +0,0 @@ -Copyright (c) 2014 Mark Maglana - -MIT License - -Permission is hereby granted, free of charge, to any person obtaining -a copy of this software and associated documentation files (the -"Software"), to deal in the Software without restriction, including -without limitation the rights to use, copy, modify, merge, publish, -distribute, sublicense, and/or sell copies of the Software, and to -permit persons to whom the Software is furnished to do so, subject to -the following conditions: - -The above copyright notice and this permission notice shall be -included in all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, -EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF -MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND -NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE -LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION -OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION -WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/aviator/Modulefile b/aviator/Modulefile deleted file mode 100644 index 133a66cb9..000000000 --- a/aviator/Modulefile +++ /dev/null @@ -1,11 +0,0 @@ -name 'aimonb-aviator' -version '0.5.1' -source 'https://github.com/aimonb/puppet_aviator' -author 'aimonb' -license 'MIT License' -summary 'Puppet feature wrapper for the Aviator OpenStack API library for Ruby' -description 'UNKNOWN' -project_page 'https://github.com/aimonb/puppet_aviator' - -## Add dependencies, if any: -# dependency 'username/name', '>= 1.2.0' diff --git a/aviator/README b/aviator/README deleted file mode 100644 index db81dbee9..000000000 --- a/aviator/README +++ /dev/null @@ -1,45 +0,0 @@ -Puppet Aviator - -A feature module for the Aviator project. - -Aviator is a lightweight library for communicating with the OpenStack -API - -See Aviator_README.md for more information on Aviator. - -License -------- -MIT License - -Contact -------- -Aimon Bustardo - -Example Usage: -------- - - $LOAD_PATH.push(File.join(File.dirname(__FILE__), '..', '..', -'..')) - require 'puppet/feature/aviator' - - configuration = { - :provider => 'openstack', - :auth_service => { - :name => 'identity', - :host_uri => 'http://devstack:5000/v2.0', - :request => 'create_token', - :validator => 'list_tenants' - }, - :auth_credentials => { - :username => 'myusername', - :password => 'mypassword', - :tenant_name => 'myproject' - } - } - - openstack = Aviator::Session.new(:config => configuration) - - openstack.authenticate - response = openstack.request :identity_service, :list_tenants, :endpoint_type => 'admin' - - puts response[:body] diff --git a/aviator/lib/puppet/feature/aviator.rb b/aviator/lib/puppet/feature/aviator.rb deleted file mode 100644 index e70ba3918..000000000 --- a/aviator/lib/puppet/feature/aviator.rb +++ /dev/null @@ -1,7 +0,0 @@ -# Add the parent dir to the load path. This is for when -# Aviator is not installed as a gem -lib_path = File.dirname(__FILE__) -$LOAD_PATH.unshift(lib_path) unless $LOAD_PATH.include? lib_path - -require 'aviator/core' -require "aviator/openstack/provider" diff --git a/aviator/lib/puppet/feature/aviator/core.rb b/aviator/lib/puppet/feature/aviator/core.rb deleted file mode 100644 index 73576f914..000000000 --- a/aviator/lib/puppet/feature/aviator/core.rb +++ /dev/null @@ -1,15 +0,0 @@ -require 'yaml' -require 'json' -require 'faraday' -require 'pathname' - -require "aviator/version" -require "aviator/core/utils/string" -require "aviator/core/utils/compatibility" -require "aviator/core/utils/hashish" -require "aviator/core/request" -require "aviator/core/request_builder" -require "aviator/core/response" -require "aviator/core/service" -require "aviator/core/session" -require "aviator/core/logger" diff --git a/aviator/lib/puppet/feature/aviator/core/cli.rb b/aviator/lib/puppet/feature/aviator/core/cli.rb deleted file mode 100644 index cd937d0e7..000000000 --- a/aviator/lib/puppet/feature/aviator/core/cli.rb +++ /dev/null @@ -1,2 +0,0 @@ -require "terminal-table" -require "aviator/core/cli/describer" diff --git a/aviator/lib/puppet/feature/aviator/core/cli/describer.rb b/aviator/lib/puppet/feature/aviator/core/cli/describer.rb deleted file mode 100644 index a6ad8507a..000000000 --- a/aviator/lib/puppet/feature/aviator/core/cli/describer.rb +++ /dev/null @@ -1,161 +0,0 @@ -module Aviator - - class Describer - - class InvalidProviderNameError < StandardError - def initialize(name) - super "Provider '#{ name }' does not exist." - end - end - - - def self.describe_aviator - str = "Available providers:\n" - - provider_names.each do |provider_name| - str << " #{ provider_name }\n" - end - - str - end - - - def self.describe_provider(provider_name) - str = "Available services for #{ provider_name }:\n" - - service_names(provider_name).each do |service_name| - str << " #{ service_name }\n" - end - - str - end - - - def self.describe_request(provider_name, service_name, api_version, endpoint_type, request_name) - service = Aviator::Service.new :provider => provider_name, :service => service_name - request_class = "Aviator::#{ provider_name.camelize }::#{ service_name.camelize }::Requests::"\ - "#{ api_version.camelize }::#{ endpoint_type.camelize }::#{ request_name.camelize }".constantize - - display = "Request: #{ request_name }\n" - - - # Build the parameters - params = request_class.optional_params.map{|p| [p, false]} + - request_class.required_params.map{|p| [p, true]} - - aliases = request_class.param_aliases - - if params.length > 0 - display << "\n" - - headings = ['NAME', 'REQUIRED?'] - - headings << 'ALIAS' if aliases.length > 0 - - rows = [] - params.sort{|a,b| a[0].to_s <=> b[0].to_s }.each do |param| - row = [ param[0], param[1] ? 'Y' : 'N' ] - - if aliases.length > 0 - row << (aliases.find{|a,p| p == param[0] } || [''])[0] - end - - rows << row - end - - widths = [ - rows.map{|row| row[0].to_s.length }.max, - rows.map{|row| row[1].to_s.length }.max - ] - - widths << rows.map{|row| row[2].to_s.length }.max if aliases.length > 0 - - table = Terminal::Table.new(:headings => headings, :rows => rows) - - table.align_column(1, :center) - - display << "Parameters:\n" - display << " " + table.to_s.split("\n").join("\n ") - display << "\n" - end - - - # Build the sample code - display << "\nSample Code:\n" - - display << " session.request(:#{ service_name }_service, :#{ request_name })" - - if params && params.length > 0 - display << " do |params|\n" - params.each do |pair| - display << " params.#{ (aliases.find{|a,p| p == pair[0] } || pair)[0] } = value\n" - end - display << " end" - end - - display << "\n" - - - # Build the links - if request_class.links && request_class.links.length > 0 - display << "\nLinks:\n" - - request_class.links.each do |link| - display << " #{ link[:rel] }:\n" - display << " #{ link[:href] }\n" - end - end - - display - end - - - def self.describe_service(provider_name, service_name) - requests = request_classes(provider_name, service_name) - - if requests.empty? - str = "No requests found for #{ provider_name } #{ service_name }_service." - else - str = "Available requests for #{ provider_name } #{ service_name }_service:\n" - - requests.each do |klass| - str << " #{ klass.api_version } #{ klass.endpoint_type } #{ klass.name.split('::').last.underscore }\n" - end - - str - end - end - - - class < provider_name, :service => service_name) - service.request_classes - end - - - def service_names(name) - provider = Pathname.new(__FILE__).join('..', '..', '..', name) - - raise InvalidProviderNameError.new(name) unless provider.exist? - - provider.children \ - .select{|c| c.directory? } \ - .map{|c| c.basename.to_s } - end - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/core/logger.rb b/aviator/lib/puppet/feature/aviator/core/logger.rb deleted file mode 100644 index bbb537d7f..000000000 --- a/aviator/lib/puppet/feature/aviator/core/logger.rb +++ /dev/null @@ -1,55 +0,0 @@ -module Aviator - - class Logger < Faraday::Response::Middleware - extend Forwardable - - def initialize(app, logger=nil) - super(app) - @logger = logger || begin - require 'logger' - ::Logger.new(self.class::LOG_FILE_PATH) - end - end - - - def_delegators :@logger, :debug, :info, :warn, :error, :fatal - - - def call(env) - info(env[:method].to_s.upcase) { env[:url].to_s } - debug('REQ_HEAD') { dump_headers env[:request_headers] } - debug('REQ_BODY') { dump_body env[:body] } - super - end - - - def on_complete(env) - info('STATUS') { env[:status].to_s } - debug('RES_HEAD') { dump_headers env[:response_headers] } - debug('RES_BODY') { dump_body env[:body] } - end - - - def self.configure(log_file_path) - # Return a subclass with its logfile path set. This - # must be done so that different sessions can log to - # different paths. - Class.new(self) { const_set('LOG_FILE_PATH', log_file_path) } - end - - - private - - def dump_body(body) - return if body.nil? - - # :TODO => Make this configurable - body.gsub(/["']password["']:["']\w*["']/, '"password":[FILTERED_VALUE]') - end - - def dump_headers(headers) - headers.map { |k, v| "#{k}: #{v.inspect}" }.join("; ") - end - end - -end diff --git a/aviator/lib/puppet/feature/aviator/core/request.rb b/aviator/lib/puppet/feature/aviator/core/request.rb deleted file mode 100644 index ed9c6cef1..000000000 --- a/aviator/lib/puppet/feature/aviator/core/request.rb +++ /dev/null @@ -1,229 +0,0 @@ -module Aviator - - class Request - - class ApiVersionNotDefinedError < StandardError - def initialize - super "api_version is not defined." - end - end - - class EndpointTypeNotDefinedError < StandardError - def initialize - super "endpoint_type is not defined." - end - end - - class PathNotDefinedError < StandardError - def initialize - super "path is not defined." - end - end - - - def initialize(session_data=nil) - @session_data = session_data - - params = self.class.params_class.new if self.class.params_class - - if params - yield(params) if block_given? - validate_params(params) - end - - @params = params - end - - - def anonymous? - self.class.anonymous? - end - - - def body? - self.class.body? - end - - - def headers? - self.class.headers? - end - - - def links - self.class.links - end - - - def optional_params - self.class.optional_params - end - - - def params - @params.dup - end - - - def required_params - self.class.required_params - end - - - def session_data - @session_data - end - - - def session_data? - !session_data.nil? - end - - - def querystring? - self.class.querystring? - end - - - def url? - self.class.url? - end - - - private - - - def validate_params(params) - required_params = self.class.required_params - - required_params.each do |name| - raise ArgumentError.new("Missing parameter #{ name }.") if params.send(name).nil? - end - end - - - # NOTE that, because we are defining the following as class methods, when they - # are called, all 'instance' variables are actually defined in the descendant class, - # not in the instance/object. This is by design since we want to keep these attributes - # within the class and because they don't change between instances anyway. - class << self - - def anonymous? - respond_to?(:anonymous) && anonymous == true - end - - - def body? - instance_methods.include? :body - end - - - def headers? - instance_methods.include? :headers - end - - - def links - @links ||= [] - end - - - def param_aliases - @param_aliases ||= {} - end - - - def params_class - all_params = required_params + optional_params - - if all_params.length > 0 && @params_class.nil? - @params_class = build_params_class(all_params, self.param_aliases) - end - - @params_class - end - - - def optional_params - @optional_params ||= [] - end - - - def querystring? - instance_methods.include? :querystring - end - - - def required_params - @required_params ||= [] - end - - - def url? - instance_methods.include? :url - end - - - private - - - def build_params_class(all_params, param_aliases) - Struct.new(*all_params) do - alias :param_getter :[] - alias :param_setter :[]= - - define_method :[] do |key| - key = param_aliases[key.to_sym] if param_aliases.keys.include? key.to_sym - param_getter(key) - end - - define_method :[]= do |key, value| - key = param_aliases[key.to_sym] if param_aliases.keys.include? key.to_sym - param_setter(key, value) - end - - param_aliases.each do |param_alias, param_name| - define_method param_alias do - param_getter(param_name) - end - - define_method "#{ param_alias }=" do |value| - param_setter(param_name, value) - end - end - end - end - - - def link(rel, href) - links << { :rel => rel, :href => href } - end - - - def meta(attr_name, attr_value) - eigenclass = class << self; self; end - eigenclass.send(:define_method, attr_name) do - attr_value - end - - define_method(attr_name) do - self.class.send(attr_name) - end - end - - - def param(param_name, opts={}) - opts = Hashish.new(opts) - list = (opts[:required] == false ? optional_params : required_params) - list << param_name unless optional_params.include?(param_name) - - if opts[:alias] - self.param_aliases[opts[:alias]] = param_name - end - end - - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/core/request_builder.rb b/aviator/lib/puppet/feature/aviator/core/request_builder.rb deleted file mode 100644 index ac5590166..000000000 --- a/aviator/lib/puppet/feature/aviator/core/request_builder.rb +++ /dev/null @@ -1,102 +0,0 @@ -module Aviator - - class BaseRequestNotFoundError < StandardError - attr_reader :base_request_hierarchy - - def initialize(base_hierarchy) - @base_request_hierarchy = base_hierarchy - super("#{ base_request_hierarchy } could not be found!") - end - end - - - class RequestAlreadyDefinedError < StandardError - attr_reader :namespace, - :request_name - - def initialize(namespace, request_name) - @namespace = namespace - @request_name = request_name - super("#{ namespace }::#{ request_name } is already defined") - end - end - - - class RequestBuilder - - class << self - - def define_request(root_namespace, request_name, options, &block) - base_klass = get_request_class(root_namespace, options[:inherit]) - - klass = Class.new(base_klass, &block) - - namespace_arr = [ - klass.provider, - klass.service, - 'Requests', - klass.api_version, - klass.endpoint_type - ] - - namespace = namespace_arr.inject(root_namespace) do |namespace, sym| - const_name = sym.to_s.camelize - namespace.const_set(const_name, Module.new) unless namespace.const_defined?(const_name, false) - namespace.const_get(const_name, false) - end - - klassname = request_name.to_s.camelize - - if namespace.const_defined?(klassname, false) - raise RequestAlreadyDefinedError.new(namespace, klassname) - end - - namespace.const_set(klassname, klass) - end - - - def get_request_class(root_namespace, request_class_arr) - provider_specific = request_class_arr != [:request] - - if provider_specific - full_request_class_arr = request_class_arr.dup - full_request_class_arr.insert(2, :requests) if provider_specific - else - full_request_class_arr = request_class_arr - end - - full_request_class_arr.inject(root_namespace) do |namespace, sym| - namespace.const_get(sym.to_s.camelize, false) - end - rescue NameError => e - if Aviator.const_defined?(full_request_class_arr[0].to_s.camelize) - provider = "Aviator::#{ full_request_class_arr[0] }::Provider".constantize - arr = ['..'] + full_request_class_arr - arr[-1,1] = arr.last.to_s + '.rb' - path = Pathname.new(provider.root_dir).join(*arr.map{|i| i.to_s }).expand_path - end - - if provider && path.exist? - require path - full_request_class_arr.inject(root_namespace) do |namespace, sym| - namespace.const_get(sym.to_s.camelize, false) - end - else - raise BaseRequestNotFoundError.new(request_class_arr) - end - end - - end - - end - - - class << self - - def define_request(request_name, options={ :inherit => [:request] }, &block) - RequestBuilder.define_request self, request_name, options, &block - end - - end # class << self - -end \ No newline at end of file diff --git a/aviator/lib/puppet/feature/aviator/core/response.rb b/aviator/lib/puppet/feature/aviator/core/response.rb deleted file mode 100644 index 12c9561af..000000000 --- a/aviator/lib/puppet/feature/aviator/core/response.rb +++ /dev/null @@ -1,52 +0,0 @@ -module Aviator - - class Response - extend Forwardable - - def_delegators :@response, :status - - attr_reader :request - - def initialize(response, request) - @response = response - @request = request - end - - - def body - @body ||= if raw_body.length > 0 - if Aviator::Compatibility::RUBY_1_8_MODE - clean_body = raw_body.gsub(/\\ /, ' ') - else - clean_body = raw_body - end - - Hashish.new(JSON.parse(clean_body)) - else - Hashish.new({}) - end - end - - - def headers - @headers ||= Hashish.new(@response.headers) - end - - - def to_hash - Hashish.new({ - :status => status, - :headers => headers, - :body => body - }) - end - - private - - def raw_body - @raw_body ||= @response.body - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/core/service.rb b/aviator/lib/puppet/feature/aviator/core/service.rb deleted file mode 100644 index f31645549..000000000 --- a/aviator/lib/puppet/feature/aviator/core/service.rb +++ /dev/null @@ -1,169 +0,0 @@ -module Aviator - - # - # Manages a service - # - class Service - - class AccessDetailsNotDefinedError < StandardError - def initialize - super ":access_details is not defined." - end - end - - class ProviderNotDefinedError < StandardError - def initialize - super ":provider is not defined." - end - end - - class ServiceNameNotDefinedError < StandardError - def initialize - super ":service is not defined." - end - end - - class SessionDataNotProvidedError < StandardError - def initialize(service_name, request_name) - super "\n\nERROR: default_session_data is not initialized and no session data was provided in\n"\ - "the method call. You have two ways to fix this:\n\n"\ - " 1) Call Session#authenticate before calling Session##{service_name}_service, or\n\n"\ - " 2) If you're really sure you don't want to authenticate beforehand,\n"\ - " construct the method call this way:\n\n"\ - " service = session.#{service_name}_service\n"\ - " service.request :#{request_name}, :api_version => :v2, :session_data => sessiondatavar\n\n"\ - " Replace :v2 with whatever available version you want to use and make sure sessiondatavar\n"\ - " is a hash that contains, at least, the :base_url key. Other keys, such as :service_token may\n"\ - " be needed depending on what the request class you are calling requires.\n\n" - end - end - - class UnknownRequestError < StandardError - def initialize(request_name, options) - super "Unknown request #{ request_name } #{ options }." - end - end - - - class MissingServiceEndpointError < StandardError - def initialize(service_name, request_name) - request_name = request_name.to_s.split('::').last.underscore - super "The session's service catalog does not have an entry for the #{ service_name } "\ - "service. Therefore, I don't know to which base URL the request should be sent. "\ - "This may be because you are using a default or unscoped token. If this is not your "\ - "intention, please authenticate with a scoped token. If using a default token is your "\ - "intention, make sure to provide a base url when you call the request. For :example => \n\n"\ - "session.#{ service_name }_service.request :#{ request_name }, :base_url => 'http://myenv.com:9999/v2.0' do |params|\n"\ - " params[:example1] = 'example1'\n"\ - " params[:example2] = 'example2'\n"\ - "end\n\n" - end - end - - attr_accessor :default_session_data - - attr_reader :service, - :provider - - - def initialize(opts={}) - @provider = opts[:provider] || (raise ProviderNotDefinedError.new) - @service = opts[:service] || (raise ServiceNameNotDefinedError.new) - @log_file = opts[:log_file] - @default_options = opts[:default_options] || {} - - @default_session_data = opts[:default_session_data] - - load_requests - end - - # - # No longer recommended for public use. Use Aviator::Session#request instead - # - def request(request_name, options={}, ¶ms) - if options[:api_version].nil? && @default_options[:api_version] - options[:api_version] = @default_options[:api_version] - end - - session_data = options[:session_data] || default_session_data - - raise SessionDataNotProvidedError.new(@service, request_name) unless session_data - - [:base_url].each do |k| - session_data[k] = options[k] if options[k] - end - - request_class = provider_module.find_request(service, request_name, session_data, options) - - raise UnknownRequestError.new(request_name, options) unless request_class - - # Always use :params over ¶ms if provided - if options[:params] - params = lambda do |params| - options[:params].each do |key, value| - begin - params[key] = value - rescue NameError => e - raise NameError.new("Unknown param name '#{key}'") - end - end - end - end - - request = request_class.new(session_data, ¶ms) - - response = http_connection.send(request.http_method) do |r| - r.url request.url - r.headers.merge!(request.headers) if request.headers? - r.query = request.querystring if request.querystring? - r.body = JSON.generate(request.body) if request.body? - end - - Aviator::Response.send(:new, response, request) - end - - - def request_classes - @request_classes - end - - - private - - - def http_connection - @http_connection ||= Faraday.new do |conn| - conn.use Logger.configure(log_file) if log_file - conn.adapter Faraday.default_adapter - - conn.headers['Content-Type'] = 'application/json' - end - end - - - def load_requests - request_file_paths = provider_module.request_file_paths(service) - request_file_paths.each{ |path| require path } - - constant_parts = request_file_paths \ - .map{|rf| rf.to_s.match(/#{provider}\/#{service}\/([\w\/]+)\.rb$/) } \ - .map{|rf| rf[1].split('/').map{|c| c.camelize }.join('::') } - - @request_classes = constant_parts.map do |cp| - "Aviator::#{provider.camelize}::#{service.camelize}::#{cp}".constantize - end - end - - - def log_file - @log_file - end - - - def provider_module - @provider_module ||= "Aviator::#{provider.camelize}::Provider".constantize - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/core/session.rb b/aviator/lib/puppet/feature/aviator/core/session.rb deleted file mode 100644 index f93db097c..000000000 --- a/aviator/lib/puppet/feature/aviator/core/session.rb +++ /dev/null @@ -1,491 +0,0 @@ -# -# Author:: Mark Maglana (mmaglana@gmail.com) -# Copyright:: Copyright (c) 2014 Mark Maglana -# License:: Distributed under the MIT license -# Homepage:: http://aviator.github.io/www/ -# -module Aviator - - # - # Manages a provider (e.g. OpenStack) session and serves as the entry point - # for a consumer class/object. See Session::new for notes on usage. - # - class Session - - class AuthenticationError < StandardError - def initialize(last_auth_body) - super("Authentication failed. The server returned #{ last_auth_body }") - end - end - - - class EnvironmentNotDefinedError < ArgumentError - def initialize(path, env) - super("The environment '#{ env }' is not defined in #{ path }.") - end - end - - class InitializationError < StandardError - def initialize - super("The session could not find :session_dump, :config_file, and " \ - ":config in the constructor arguments provided") - end - end - - class InvalidConfigFilePathError < ArgumentError - def initialize(path) - super("The config file at #{ path } does not exist!") - end - end - - - class NotAuthenticatedError < StandardError - def initialize - super("Session is not authenticated. Please authenticate before proceeding.") - end - end - - - class ValidatorNotDefinedError < StandardError - def initialize - super("The validator request name is not defined for this session object.") - end - end - - # - # Create a new Session instance. - # - # Initialize with a config file - # - # Aviator::Session.new(:config_file => 'path/to/aviator.yml', :environment => :production) - # - # In the above example, the config file must have the following form: - # - # production: - # provider: openstack - # auth_service: - # name: identity - # host_uri: 'http://my.openstackenv.org:5000' - # request: create_token - # validator: list_tenants - # api_version: v2 - # auth_credentials: - # username: myusername - # password: mypassword - # tenant_name: myproject - # - # SIDENOTE: For more information about the validator member, see Session#validate. - # - # Once the session has been instantiated, you may authenticate against the - # provider as follows: - # - # session.authenticate - # - # The members you put under auth_credentials will depend on the request - # class you declare under auth_service:request and what parameters it - # accepts. To know more about a request class and its parameters, you can use - # the CLI tool aviator describe or view the request definition file directly. - # - # If writing the auth_credentials in the config file is not acceptable, - # you may omit it and just supply the credentials at runtime. For example: - # - # session.authenticate do |params| - # params.username = ARGV[0] - # params.password = ARGV[1] - # params.tenant_name = ARGV[2] - # end - # - # See Session#authenticate for more info. - # - # Note that while the example config file above only has one environment (production), - # you can declare an arbitrary number of environments in your config file. Shifting - # between environments is as simple as changing the :environment to refer to that. - # - # - # Initialize with an in-memory hash - # - # You can create an in-memory hash with a structure similar to the config file but without - # the environment name. For example: - # - # configuration = { - # :provider => 'openstack', - # :auth_service => { - # :name => 'identity', - # :host_uri => 'http://devstack:5000/v2.0', - # :request => 'create_token', - # :validator => 'list_tenants' - # } - # } - # - # Supply this to the initializer using the :config option. For example: - # - # Aviator::Session.new(:config => configuration) - # - # - # Initialize with a session dump - # - # You can create a new Session instance using a dump from another instance. For example: - # - # session_dump = session1.dump - # session2 = Aviator::Session.new(:session_dump => session_dump) - # - # However, Session.load is cleaner and recommended over this method. - # - # - # Optionally supply a log file - # - # In all forms above, you may optionally add a :log_file option to make - # Aviator write all HTTP calls to the given path. For example: - # - # Aviator::Session.new(:config_file => 'path/to/aviator.yml', :environment => :production, :log_file => 'path/to/log') - # - def initialize(opts={}) - if opts.has_key? :session_dump - initialize_with_dump(opts[:session_dump]) - elsif opts.has_key? :config_file - initialize_with_config(opts[:config_file], opts[:environment]) - elsif opts.has_key? :config - initialize_with_hash(opts[:config]) - else - raise InitializationError.new - end - - @log_file = opts[:log_file] - end - - # - # Authenticates against the backend provider using the auth_service request class - # declared in the session's configuration. Please see Session.new for more information - # on declaring the request class to use for authentication. - # - # Request params block - # - # If the auth_service request class accepts parameters, you may supply that - # as a block and it will be directly passed to the request. For example: - # - # session = Aviator::Session.new(:config => config) - # session.authenticate do |params| - # params.username = username - # params.password = password - # params.tenant_name = project - # end - # - # If your configuration happens to have an auth_credentials in it, those - # will be overridden by this block. - # - # Treat parameters as a hash - # - # You can also treat the params struct like a hash with the attribute - # names as the keys. For example, we can rewrite the above as: - # - # session = Aviator::Session.new(:config => config) - # session.authenticate do |params| - # params[:username] = username - # params[:password] = password - # params[:tenant_name] = project - # end - # - # Keys can be symbols or strings. - # - # Use a hash argument instead of a block - # - # You may also provide request params as an argument instead of a block. This is - # especially useful if you want to mock Aviator as it's easier to specify ordinary - # argument expectations over blocks. Further rewriting the example above, - # we end up with: - # - # session = Aviator::Session.new(:config => config) - # session.authenticate :params => { - # :username => username, - # :password => password, - # :tenant_name => project - # } - # - # If both :params and a block are provided, the :params - # values will be used and the block ignored. - # - # Success requirements - # - # Expects an HTTP status 200 or 201 response from the backend. Any other - # status is treated as a failure. - # - def authenticate(opts={}, &block) - block ||= lambda do |params| - config[:auth_credentials].each do |key, value| - begin - params[key] = value - rescue NameError => e - raise NameError.new("Unknown param name '#{key}'") - end - end - end - - response = auth_service.request(config[:auth_service][:request].to_sym, opts, &block) - - if [200, 201].include? response.status - @auth_response = Hashish.new({ - :headers => response.headers, - :body => response.body - }) - update_services_session_data - else - raise AuthenticationError.new(response.body) - end - self - end - - # - # Returns true if the session has been authenticated. Note that this relies on - # cached response from a previous run of Session#authenticate if one was made. - # If you want to check against the backend provider if the session is still valid, - # use Session#validate instead. - # - def authenticated? - !auth_response.nil? - end - - # - # Returns its configuration. - # - def config - @config - end - - # - # Returns a JSON string of its configuration and auth_data. This string can be streamed - # or stored and later re-loaded in another Session instance. For example: - # - # session = Aviator::Session.new(:config => configuration) - # str = session.dump - # - # # time passes... - # - # session = Aviator::Session.load(str) - # - def dump - JSON.generate({ - :config => config, - :auth_response => auth_response - }) - end - - - # - # Same as Session::load but re-uses the Session instance this method is - # called on instead of creating a new one. - # - def load(session_dump) - initialize_with_dump(session_dump) - update_services_session_data - self - end - - - def method_missing(name, *args, &block) # :nodoc: - service_name_parts = name.to_s.match(/^(\w+)_service$/) - - if service_name_parts - get_service_obj(service_name_parts[1]) - else - super name, *args, &block - end - end - - - # - # Creates a new Session object from a previous session's dump. See Session#dump for - # more information. - # - # If you want the newly deserialized session to log its output, add a :log_file - # option. - # - # Aviator::Session.load(session_dump_str, :log_file => 'path/to/aviator.log') - # - def self.load(session_dump, opts={}) - opts[:session_dump] = session_dump - - new(opts) - end - - - # - # Returns the log file path. May be nil if none was provided during initialization. - # - def log_file - @log_file - end - - - # - # Calls the given request of the given service. An example call might look like: - # - # session.request :compute_service, :create_server do |p| - # p.name = "My Server" - # p.image_ref = "7cae8c8e-fb01-4a88-bba3-ae0fcb1dbe29" - # p.flavor_ref = "fa283da1-59a5-4245-8569-b6eadf69f10b" - # end - # - # Note that you can also treat the block's argument like a hash with the attribute - # names as the keys. For example, we can rewrite the above as: - # - # session.request :compute_service, :create_server do |p| - # p[:name] = "My Server" - # p[:image_ref] = "7cae8c8e-fb01-4a88-bba3-ae0fcb1dbe29" - # p[:flavor_ref] = "fa283da1-59a5-4245-8569-b6eadf69f10b" - # end - # - # Keys can be symbols or strings. - # - # You may also provide parameters as an argument instead of a block. This is - # especially useful when mocking Aviator as it's easier to specify ordinary - # argument expectations over blocks. Further rewriting the example above, - # we end up with: - # - # session.request :compute_service, :create_server, :params => { - # :name => "My Server", - # :image_ref => "7cae8c8e-fb01-4a88-bba3-ae0fcb1dbe29", - # :flavor_ref => "fa283da1-59a5-4245-8569-b6eadf69f10b" - # } - # - # If both :params and a block are provided, the values in :params - # will be used and the block ignored. - # - # Return Value - # - # The return value will be an instance of Hashish, a lightweight replacement for - # activesupport's HashWithIndifferentAccess, with the following structure: - # - # { - # :status => 200, - # :headers => { - # 'X-Auth-Token' => 'd9186f45ce5446eaa0adc9def1c46f5f', - # 'Content-Type' => 'application/json' - # }, - # :body => { - # :some_key => :some_value - # } - # } - # - # Note that the members in :headers and :body will vary depending - # on the provider and the request that was made. - # - # --- - # - # Request Options - # - # You can further customize how the method behaves by providing one or more - # options to the call. For example, assuming you are using the openstack - # provider, the following will call the :create_server request of the - # v1 API of :compute_service. - # - # session.request :compute_service, :create_server, :api_version => v1, :params => params - # - # The available options vary depending on the provider. See the documentation - # on the provider's Provider class for more information (e.g. Aviator::Openstack::Provider) - # - def request(service_name, request_name, opts={}, &block) - service = send("#{service_name.to_s}_service") - response = service.request(request_name, opts, &block) - response.to_hash - end - - - # - # Returns true if the session is still valid in the underlying provider. This method calls - # the validator request class declared under auth_service in the - # configuration. The validator can be any request class as long as: - # - # * The request class exists! - # * Is not an anonymous request. Otherwise it will always return true. - # * Does not require any parameters - # * It returns an HTTP status 200 or 203 to indicate auth info validity. - # * It returns any other HTTP status to indicate that the auth info is invalid. - # - # See Session::new for an example on how to specify the request class to use for session validation. - # - # Note that this method requires the session to be previously authenticated otherwise a - # NotAuthenticatedError will be raised. If you just want to check if the session was previously - # authenticated, use Session#authenticated? instead. - # - def validate - raise NotAuthenticatedError.new unless authenticated? - raise ValidatorNotDefinedError.new unless config[:auth_service][:validator] - - auth_with_bootstrap = auth_response.merge({ :auth_service => config[:auth_service] }) - - response = auth_service.request config[:auth_service][:validator].to_sym, :session_data => auth_with_bootstrap - response.status == 200 || response.status == 203 - end - - - private - - - def auth_response - @auth_response - end - - - def auth_service - @auth_service ||= Service.new( - :provider => config[:provider], - :service => config[:auth_service][:name], - :default_session_data => { :auth_service => config[:auth_service] }, - :log_file => log_file - ) - end - - - def get_service_obj(service_name) - @services ||= {} - - if @services[service_name].nil? - default_options = config["#{ service_name }_service"] - - @services[service_name] = Service.new( - :provider => config[:provider], - :service => service_name, - :default_session_data => auth_response, - :default_options => default_options, - :log_file => log_file - ) - end - - @services[service_name] - end - - - def initialize_with_config(config_path, environment) - raise InvalidConfigFilePathError.new(config_path) unless Pathname.new(config_path).file? - - all_config = Hashish.new(YAML.load_file(config_path)) - - raise EnvironmentNotDefinedError.new(config_path, environment) unless all_config[environment] - - @config = all_config[environment] - end - - - def initialize_with_dump(session_dump) - session_info = Hashish.new(JSON.parse(session_dump)) - @config = session_info[:config] - @auth_response = session_info[:auth_response] - end - - - def initialize_with_hash(hash_obj) - @config = Hashish.new(hash_obj) - end - - - def update_services_session_data - return unless @services - - @services.each do |name, obj| - obj.default_session_data = auth_response - end - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/core/utils/compatibility.rb b/aviator/lib/puppet/feature/aviator/core/utils/compatibility.rb deleted file mode 100644 index 560fc2b70..000000000 --- a/aviator/lib/puppet/feature/aviator/core/utils/compatibility.rb +++ /dev/null @@ -1,34 +0,0 @@ -module Aviator - - module Compatibility - RUBY_1_8_MODE = (not (RUBY_VERSION =~ /1\.8\.\d*/).nil?) - end - -end - -if Aviator::Compatibility::RUBY_1_8_MODE - - class Module - - alias_method :old_const_defined?, :const_defined? - - def const_defined?(sym, ignore=nil) - old_const_defined?(sym) - end - - - alias_method :old_const_get, :const_get - - def const_get(sym, ignore=nil) - old_const_get(sym) - end - - alias_method :old_instance_methods, :instance_methods - - def instance_methods(include_super=true) - old_instance_methods(include_super).map(&:to_sym) - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/core/utils/hashish.rb b/aviator/lib/puppet/feature/aviator/core/utils/hashish.rb deleted file mode 100644 index b50b5bf92..000000000 --- a/aviator/lib/puppet/feature/aviator/core/utils/hashish.rb +++ /dev/null @@ -1,127 +0,0 @@ -require 'json' - -# Hash-ish! -# -# This class is implemented using composition rather than inheritance so -# that we have control over what operations it exposes to peers. -class Hashish - include Enumerable - - def initialize(hash={}) - @hash = hash.dup - stringify_keys - hashishify_values - end - - def ==(other_obj) - other_obj.class == self.class && - other_obj.hash == self.hash - end - - def [](key) - @hash[normalize(key)] - end - - def []=(key, value) - @hash[normalize(key)] = value - end - - def dup - Hashish.new(JSON.parse(@hash.to_json)) - end - - def each(&block) - @hash.each(&block) - end - - def empty? - @hash.empty? - end - - def has_key?(name) - @hash.has_key? normalize(name) - end - - def hash - @hash - end - - def keys - @hash.keys - end - - def length - @hash.length - end - - def merge(other_hash) - Hashish.new(@hash.merge(other_hash)) - end - - def merge!(other_hash) - @hash.merge! other_hash - self - end - - def to_json(obj) - @hash.to_json(obj) - end - - def to_s - str = "{" - @hash.each do |key, value| - if value.kind_of? String - value = "'#{value}'" - elsif value.nil? - value = "nil" - elsif value.kind_of? Array - value = "[#{value.join(", ")}]" - end - - str += " #{key}: #{value}," - end - - str = str[0...-1] + " }" - str - end - - private - - # Hashishify all the things! - def hashishify_values - @hash.each do |key, value| - if @hash[key].kind_of? Hash - @hash[key] = Hashish.new(value) - elsif @hash[key].kind_of? Array - @hash[key].each_index do |index| - element = @hash[key][index] - if element.kind_of? Hash - @hash[key][index] = Hashish.new(element) - end - end - end - end - end - - - def normalize(key) - if @hash.has_key? key - key - elsif key.is_a? Symbol - key.to_s - else - key - end - end - - - def stringify_keys - keys = @hash.keys - keys.each do |key| - if key.is_a? Symbol - @hash[key.to_s] = @hash.delete(key) - end - end - end - -end diff --git a/aviator/lib/puppet/feature/aviator/core/utils/string.rb b/aviator/lib/puppet/feature/aviator/core/utils/string.rb deleted file mode 100644 index 25d24b66e..000000000 --- a/aviator/lib/puppet/feature/aviator/core/utils/string.rb +++ /dev/null @@ -1,24 +0,0 @@ -class String - - unless instance_methods.include? 'camelize' - define_method :camelize do - word = self.slice(0,1).capitalize + self.slice(1..-1) - word.gsub(/_([a-zA-Z\d])/) { "#{$1.capitalize}" } - end - end - - unless instance_methods.include? 'constantize' - define_method :constantize do - self.split("::").inject(Object) do |namespace, sym| - namespace.const_get(sym.to_s.camelize, false) - end - end - end - - unless instance_methods.include? 'underscore' - define_method :underscore do - self.gsub(/([a-z\d])([A-Z])/, '\1_\2').downcase - end - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/common/requests/v0/public/base.rb b/aviator/lib/puppet/feature/aviator/openstack/common/requests/v0/public/base.rb deleted file mode 100644 index a80b8c850..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/common/requests/v0/public/base.rb +++ /dev/null @@ -1,125 +0,0 @@ -module Aviator - - define_request :base do - - meta :provider, :openstack - meta :service, :common - meta :api_version, :v0 - meta :endpoint_type, :public - - def headers - {}.tap do |h| - if self.anonymous? - # do nothing - - elsif session_data.has_key? :service_token - # service_token is the token that one would bootstrap OpenStack - # with during the installation process. This token can be used - # directly and does not require authentication beforehand. - h['X-Auth-Token'] = session_data[:service_token] - - elsif keystone_v2_style_session_data? - h['X-Auth-Token'] = session_data[:body][:access][:token][:id] - - elsif keystone_v3_style_session_data? - h['X-Auth-Token'] = session_data[:headers]['x-subject-token'] - - else - raise "Unknown session data format!" - - end - end - end - - - private - - def base_url - if session_data[:base_url] - session_data[:base_url] - - elsif keystone_v2_style_session_data? && keystone_v2_style_service_info? - extract_base_url_from_keystone_v2_session_data - - elsif keystone_v3_style_session_data? && keystone_v3_style_service_info? - extract_base_url_from_keystone_v3_session_data - - elsif session_data[:auth_service] && session_data[:auth_service][:host_uri] && session_data[:auth_service][:api_version] - version = session_data[:auth_service][:api_version].to_s == "v2" ? "v2.0" : session_data[:auth_service][:api_version] - "#{ session_data[:auth_service][:host_uri] }/#{ version }" - - elsif session_data[:auth_service] && session_data[:auth_service][:host_uri] - session_data[:auth_service][:host_uri] - - else - raise Aviator::Service::MissingServiceEndpointError.new(service.to_s, self.class) - end - end - - - def build_service_type_string - api_versions_without_a_suffix = [ - [:compute, :v2], - [:ec2, :v1], - [:identity, :v2], - [:image, :v1], - [:metering, :v1], - [:s3, :v1], - [:volume, :v1] - ] - - if api_versions_without_a_suffix.include? [service, api_version] - service.to_s - else - "#{ service }#{ api_version }" - end - end - - - def extract_base_url_from_keystone_v2_session_data - service_info = session_data[:body][:access][:serviceCatalog].find{ |s| s[:type] == build_service_type_string } - service_info[:endpoints][0]["#{ endpoint_type }URL".to_sym] - end - - - def extract_base_url_from_keystone_v3_session_data - service_info = session_data[:body][:token][:catalog].select{ |s| s[:type] == build_service_type_string } - endpoints = service_info.find{ |s| s.keys.include? "endpoints" }['endpoints'] - - endpoints.find{ |s| s['interface'] == endpoint_type.to_s }['url'] - end - - - def keystone_v2_style_service_info? - not session_data[:body][:access][:serviceCatalog].find{ |s| s[:type] == build_service_type_string }.nil? - end - - - def keystone_v2_style_session_data? - session_data.has_key?(:body) && session_data[:body].has_key?(:access) - end - - - def keystone_v3_style_service_info? - not session_data[:body][:token][:catalog].find{ |s| s[:type] == build_service_type_string }.nil? - end - - - def keystone_v3_style_session_data? - session_data.has_key?(:headers) && session_data[:headers].has_key?("x-subject-token") - end - - - def params_to_querystring(param_names) - filters = [] - - param_names.each do |param_name| - filters << "#{ param_name }=#{ params[param_name] }" if params[param_name] - end - - filters.empty? ? "" : "?#{ filters.join('&') }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/common/requests/v2/admin/base.rb b/aviator/lib/puppet/feature/aviator/openstack/common/requests/v2/admin/base.rb deleted file mode 100644 index dd07d2f31..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/common/requests/v2/admin/base.rb +++ /dev/null @@ -1,9 +0,0 @@ -module Aviator - - define_request :base, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :endpoint_type, :admin - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/common/requests/v2/public/base.rb b/aviator/lib/puppet/feature/aviator/openstack/common/requests/v2/public/base.rb deleted file mode 100644 index 863a1ddc8..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/common/requests/v2/public/base.rb +++ /dev/null @@ -1,9 +0,0 @@ -module Aviator - - define_request :base, :inherit => [:openstack, :common, :v0, :public, :base] do - - meta :api_version, :v2 - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/common/requests/v3/public/base.rb b/aviator/lib/puppet/feature/aviator/openstack/common/requests/v3/public/base.rb deleted file mode 100644 index 1340aeab0..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/common/requests/v3/public/base.rb +++ /dev/null @@ -1,9 +0,0 @@ -module Aviator - - define_request :base, :inherit => [:openstack, :common, :v0, :public, :base] do - - meta :api_version, :v3 - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/confirm_server_resize.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/confirm_server_resize.rb deleted file mode 100644 index 0c672da8a..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/confirm_server_resize.rb +++ /dev/null @@ -1,36 +0,0 @@ -module Aviator - - define_request :confirm_server_resize, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Confirm_Resized_Server-d1e3868.html' - - param :id, :required => true - - - def body - { - :confirmResize => nil - } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/create_network.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/create_network.rb deleted file mode 100644 index f45ef1a78..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/create_network.rb +++ /dev/null @@ -1,56 +0,0 @@ -module Aviator - - define_request :create_network, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :compute - - link 'documentation', - 'http://api.openstack.org/api-ref-compute.html#ext-os-networks' - - - param :label, :required => true - param :bridge, :required => false - param :bridge_interface, :required => false - param :cidr, :required => false - param :cidr_v6, :required => false - param :dns1, :required => false - param :dns2, :required => false - param :gateway, :required => false - param :gateway_v6, :required => false - param :multi_host, :required => false - param :project_id, :required => false - param :vlan, :required => false - - - def body - p = { - :network => { - :label => params[:label] - } - } - - optional_params.each do |key| - p[:network][key] = params[key] if params[key] - end - - p - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/os-networks" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/get_host_details.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/get_host_details.rb deleted file mode 100644 index daf25a3cd..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/get_host_details.rb +++ /dev/null @@ -1,29 +0,0 @@ -module Aviator - - define_request :get_host_details, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :compute - - link 'documentation', - 'http://api.openstack.org/api-ref.html#ext-os-hosts' - - param :host_name, :required => true - - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/os-hosts/#{ params[:host_name] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/list_hosts.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/list_hosts.rb deleted file mode 100644 index 8c47aa835..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/list_hosts.rb +++ /dev/null @@ -1,44 +0,0 @@ -module Aviator - - define_request :list_hosts, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :compute - - link 'documentation', - 'http://api.openstack.org/api-ref.html#ext-os-hosts' - - link 'documentation bug', - 'https://bugs.launchpad.net/nova/+bug/1224763' - - param :service, :required => false - param :zone, :required => false - - - def headers - super - end - - - def http_method - :get - end - - - def url - url = "#{ base_url }/os-hosts" - - filters = [] - - optional_params.each do |param_name| - filters << "#{ param_name }=#{ params[param_name] }" if params[param_name] - end - - url += "?#{ filters.join('&') }" unless filters.empty? - - url - end - - end - -end - diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/list_hypervisors.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/list_hypervisors.rb deleted file mode 100644 index 5f84de5df..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/list_hypervisors.rb +++ /dev/null @@ -1,27 +0,0 @@ -module Aviator - - define_request :list_hypervisors, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :compute - - link 'documentation', - 'http://api.openstack.org/api-ref-compute-v2-ext.html#ext-os-hypervisors' - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/os-hypervisors" - end - - end - -end - diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/lock_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/lock_server.rb deleted file mode 100644 index 6db89fee2..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/lock_server.rb +++ /dev/null @@ -1,34 +0,0 @@ -module Aviator - - define_request :lock_server, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/POST_lock_v2__tenant_id__servers__server_id__action_ext-os-admin-actions.html' - - param :id, :required => true - - - def body - { :lock => nil } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/migrate_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/migrate_server.rb deleted file mode 100644 index 56259cdff..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/migrate_server.rb +++ /dev/null @@ -1,34 +0,0 @@ -module Aviator - - define_request :migrate_server, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/POST_migrate_v2__tenant_id__servers__server_id__action_ext-os-admin-actions.html' - - param :id, :required => true - - - def body - { :migrate => nil } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/reset_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/reset_server.rb deleted file mode 100644 index ea8af1f0a..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/reset_server.rb +++ /dev/null @@ -1,39 +0,0 @@ -module Aviator - - define_request :reset_server, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/POST_os-resetState_v2__tenant_id__servers__server_id__action_ext-os-admin-actions.html' - - param :id, :required => true - param :state, :required => true - - - def body - { - 'os-resetState' => { - 'state' => params[:state] - } - } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/resize_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/resize_server.rb deleted file mode 100644 index e4b6582a6..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/resize_server.rb +++ /dev/null @@ -1,41 +0,0 @@ -module Aviator - - define_request :resize_server, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Resize_Server-d1e3707.html' - - param :id, :required => true - param :name, :required => true - param :flavorRef, :required => true, :alias => :flavor_ref - - - def body - { - :resize => { - :name => params[:name], - :flavorRef => params[:flavorRef] - } - } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/revert_server_resize.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/revert_server_resize.rb deleted file mode 100644 index 3eb95164b..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/revert_server_resize.rb +++ /dev/null @@ -1,46 +0,0 @@ -module Aviator - - define_request :revert_server_resize do - - meta :provider, :openstack - meta :service, :compute - meta :api_version, :v2 - meta :endpoint_type, :admin - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Revert_Resized_Server-d1e4024.html' - - param :id, :required => true - - - def body - { - :revertResize => nil - } - end - - - def headers - h = {} - - unless self.anonymous? - h['X-Auth-Token'] = session_data[:body][:access][:token][:id] - end - - h - end - - - def http_method - :post - end - - - def url - service_spec = session_data[:body][:access][:serviceCatalog].find{|s| s[:type] == service.to_s } - "#{ service_spec[:endpoints][0][:adminURL] }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/unlock_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/unlock_server.rb deleted file mode 100644 index eede63524..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/admin/unlock_server.rb +++ /dev/null @@ -1,34 +0,0 @@ -module Aviator - - define_request :unlock_server, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/POST_unlock_v2__tenant_id__servers__server_id__action_ext-os-admin-actions.html' - - param :id, :required => true - - - def body - { :unlock => nil } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/add_floating_ip.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/add_floating_ip.rb deleted file mode 100644 index 56ff62457..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/add_floating_ip.rb +++ /dev/null @@ -1,38 +0,0 @@ -module Aviator - - define_request :add_floating_ip, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/POST_os-floating-ips-v2_AddFloatingIP__v2__tenant_id__servers__server_id__action_ext-os-floating-ips.html#POST_os-floating-ips-v2_AddFloatingIP__v2__tenant_id__servers__server_id__action_ext-os-floating-ips-Request' - - param :server_id, :required => true - param :address, :required => true - - def body - { - :addFloatingIp => { - :address => params[:address] - } - } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:server_id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/allocate_floating_ip.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/allocate_floating_ip.rb deleted file mode 100644 index 92b9c5c37..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/allocate_floating_ip.rb +++ /dev/null @@ -1,35 +0,0 @@ -module Aviator - - define_request :allocate_floating_ip, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/POST_os-floating-ips-v2_AllocateFloatingIP__v2__tenant_id__os-floating-ips_ext-os-floating-ips.html' - - param :pool, :required => false - - def body - { - :pool => params[:pool] - } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/os-floating-ips" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/change_admin_password.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/change_admin_password.rb deleted file mode 100644 index 03aced50a..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/change_admin_password.rb +++ /dev/null @@ -1,44 +0,0 @@ -module Aviator - - define_request :change_admin_password, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Change_Password-d1e3234.html' - - link 'additional spec', - 'https://answers.launchpad.net/nova/+question/228462' - - param :adminPass, :required => true, :alias => :admin_pass - param :id, :required => true - - - def body - p = { - :changePassword => { - :adminPass => params[:adminPass] - } - } - - p - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_image.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_image.rb deleted file mode 100644 index 0a4622370..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_image.rb +++ /dev/null @@ -1,46 +0,0 @@ -module Aviator - - define_request :create_image, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Create_Image-d1e4655.html' - - param :id, :required => true - param :metadata, :required => false - param :name, :required => true - - - def body - p = { - :createImage => { - :name => params[:name] - } - } - - [:metadata].each do |key| - p[:createImage][key] = params[key] if params[key] - end - - p - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_keypair.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_keypair.rb deleted file mode 100644 index b90485fb4..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_keypair.rb +++ /dev/null @@ -1,39 +0,0 @@ -module Aviator - - define_request :create_keypair, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/POST_os-keypairs-v2_createKeypair__v2__tenant_id__os-keypairs_ext-os-keypairs.html' - - param :name, :required => true - param :public_key, :required => false - - def body - { - :keypair => { - :name => params[:name], - :public_key => params[:public_key], - } - } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/os-keypairs" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_server.rb deleted file mode 100644 index 682223543..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/create_server.rb +++ /dev/null @@ -1,55 +0,0 @@ -module Aviator - - define_request :create_server, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/CreateServers.html' - - param :accessIPv4, :required => false, :alias => :access_ipv4 - param :accessIPv6, :required => false, :alias => :access_ipv6 - param :adminPass, :required => false, :alias => :admin_pass - param :imageRef, :required => true, :alias => :image_ref - param :flavorRef, :required => true, :alias => :flavor_ref - param :key_name, :required => false - param :metadata, :required => false - param :name, :required => true - param :networks, :required => false - param :personality, :required => false - - - def body - p = { - :server => { - :flavorRef => params[:flavorRef], - :imageRef => params[:imageRef], - :name => params[:name] - } - } - - [:adminPass, :metadata, :personality, :networks, :accessIPv4, :accessIPv6, :key_name].each do |key| - p[:server][key] = params[key] if params[key] - end - - p - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_image.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_image.rb deleted file mode 100644 index 35ea8cf40..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_image.rb +++ /dev/null @@ -1,29 +0,0 @@ -module Aviator - - define_request :delete_image, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Delete_Image-d1e4957.html' - - param :id, :required => true - - - def headers - super - end - - - def http_method - :delete - end - - - def url - "#{ base_url }/images/#{ params[:id]}" - end - - end - -end \ No newline at end of file diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_image_metadata_item.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_image_metadata_item.rb deleted file mode 100644 index 7b1fc3d70..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_image_metadata_item.rb +++ /dev/null @@ -1,31 +0,0 @@ -module Aviator - - define_request :delete_image_metadata_item, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Delete_Metadata_Item-d1e5790.html' - - - param :id, :required => true - param :key, :required => true - - - def headers - super - end - - - def http_method - :delete - end - - - def url - "#{ base_url }/images/#{ params[:id] }/metadata/#{ params[:key] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_server.rb deleted file mode 100644 index a991efb8e..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_server.rb +++ /dev/null @@ -1,29 +0,0 @@ -module Aviator - - define_request :delete_server, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Delete_Server-d1e2883.html' - - param :id, :required => true - - - def headers - super - end - - - def http_method - :delete - end - - - def url - "#{ base_url }/servers/#{ params[:id] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_server_metadata_item.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_server_metadata_item.rb deleted file mode 100644 index facbbb537..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/delete_server_metadata_item.rb +++ /dev/null @@ -1,31 +0,0 @@ -module Aviator - - define_request :delete_server_metadata_item, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Delete_Metadata_Item-d1e5790.html' - - - param :id, :required => true - param :key, :required => true - - - def headers - super - end - - - def http_method - :delete - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/metadata/#{ params[:key] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_flavor_details.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_flavor_details.rb deleted file mode 100644 index 8faeb900e..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_flavor_details.rb +++ /dev/null @@ -1,29 +0,0 @@ -module Aviator - - define_request :get_flavor_details, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Get_Flavor_Details-d1e4317.html' - - param :id, :required => true - - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/flavors/#{ params[:id] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_image_details.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_image_details.rb deleted file mode 100644 index b9fd3cce8..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_image_details.rb +++ /dev/null @@ -1,29 +0,0 @@ -module Aviator - - define_request :get_image_details, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Get_Image_Details-d1e4848.html' - - param :id, :required => true - - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/images/#{ params[:id]}" - end - - end - -end \ No newline at end of file diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_image_metadata_item.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_image_metadata_item.rb deleted file mode 100644 index 01476744a..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_image_metadata_item.rb +++ /dev/null @@ -1,31 +0,0 @@ -module Aviator - - define_request :get_image_metadata_item, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Get_Metadata_Item-d1e5507.html' - - - param :id, :required => true - param :key, :required => true - - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/images/#{ params[:id] }/metadata/#{ params[:key] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_network_details.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_network_details.rb deleted file mode 100644 index 8d4192bfe..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_network_details.rb +++ /dev/null @@ -1,30 +0,0 @@ -module Aviator - - define_request :get_network_details, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://api.openstack.org/api-ref-compute.html#ext-os-networks' - - - param :id, :required => true - - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/os-networks/#{ params[:id] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_server.rb deleted file mode 100644 index e5c391106..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_server.rb +++ /dev/null @@ -1,28 +0,0 @@ -module Aviator - - define_request :get_server, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Get_Server_Details-d1e2623.html' - - param :id, :required => true - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/servers/#{ params[:id] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_server_metadata_item.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_server_metadata_item.rb deleted file mode 100644 index 580320f24..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/get_server_metadata_item.rb +++ /dev/null @@ -1,31 +0,0 @@ -module Aviator - - define_request :get_server_metadata_item, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Get_Metadata_Item-d1e5507.html' - - - param :id, :required => true - param :key, :required => true - - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/metadata/#{ params[:key] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_addresses.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_addresses.rb deleted file mode 100644 index 2c6fe60a4..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_addresses.rb +++ /dev/null @@ -1,36 +0,0 @@ -module Aviator - - define_request :list_addresses, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/List_Addresses-d1e3014.html' - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/List_Addresses_by_Network-d1e3118.html' - - - param :id, :required => true - param :networkID, :required => false, :alias => :network_id - - - def headers - super - end - - - def http_method - :get - end - - - def url - url = "#{ base_url }/servers/#{ params[:id] }/ips" - url += "/#{ params[:networkID] }" if params[:networkID] - url - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_flavors.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_flavors.rb deleted file mode 100644 index 9eba13626..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_flavors.rb +++ /dev/null @@ -1,35 +0,0 @@ -module Aviator - - define_request :list_flavors, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/List_Flavors-d1e4188.html' - - param :details, :required => false - param :minDisk, :required => false, :alias => :min_disk - param :minRam, :required => false, :alias => :min_ram - param :marker, :required => false - param :limit, :required => false - - - def headers - super - end - - - def http_method - :get - end - - - def url - str = "#{ base_url }/flavors" - str += "/detail" if params[:details] - str += params_to_querystring(optional_params + required_params - [:details]) - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_floating_ips.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_floating_ips.rb deleted file mode 100644 index 54f07be3a..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_floating_ips.rb +++ /dev/null @@ -1,26 +0,0 @@ -module Aviator - - define_request :list_floating_ips, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/GET_os-floating-ips-v2_ListFloatingIPs__v2__tenant_id__os-floating-ips_ext-os-floating-ips.html' - - def headers - super - end - - - def http_method - :get - end - - - def url - str = "#{ base_url }/os-floating-ips" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_image_metadata.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_image_metadata.rb deleted file mode 100644 index 7b176891f..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_image_metadata.rb +++ /dev/null @@ -1,30 +0,0 @@ -module Aviator - - define_request :list_image_metadata, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/List_Metadata-d1e5089.html' - - - param :id, :required => true - - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/images/#{ params[:id] }/metadata" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_images.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_images.rb deleted file mode 100644 index b282aec97..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_images.rb +++ /dev/null @@ -1,38 +0,0 @@ -module Aviator - - define_request :list_images, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/List_Images-d1e4435.html' - - param :details, :required => false - param :server, :required => false - param :name, :required => false - param :status, :required => false - param 'changes-since', :required => false, :alias => :changes_since - param :marker, :required => false - param :limit, :required => false - param :type, :required => false - - - def headers - super - end - - - def http_method - :get - end - - - def url - str = "#{ base_url }/images" - str += "/detail" if params[:details] - str += params_to_querystring(optional_params + required_params - [:details]) - end - - end - -end \ No newline at end of file diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_keypairs.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_keypairs.rb deleted file mode 100644 index f01ce0bd6..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_keypairs.rb +++ /dev/null @@ -1,26 +0,0 @@ -module Aviator - - define_request :list_keypairs, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/GET_os-keypairs-v2_getKeypair__v2__tenant_id__os-keypairs_ext-os-keypairs.html' - - def headers - super - end - - - def http_method - :get - end - - - def url - str = "#{ base_url }/os-keypairs" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_networks.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_networks.rb deleted file mode 100644 index 472ab1bd0..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_networks.rb +++ /dev/null @@ -1,27 +0,0 @@ -module Aviator - - define_request :list_networks, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://api.openstack.org/api-ref-compute.html#ext-os-networks' - - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/os-networks" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_server_metadata.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_server_metadata.rb deleted file mode 100644 index d90936c1b..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_server_metadata.rb +++ /dev/null @@ -1,30 +0,0 @@ -module Aviator - - define_request :list_server_metadata, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/List_Metadata-d1e5089.html' - - - param :id, :required => true - - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/metadata" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_servers.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_servers.rb deleted file mode 100644 index f8126f0c5..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/list_servers.rb +++ /dev/null @@ -1,58 +0,0 @@ -module Aviator - - define_request :list_servers, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/List_Servers-d1e2078.html' - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/GET_listServers_v2__tenant_id__servers_compute_servers.html' - - link 'github :issue => getting all servers', - 'https://github.com/aviator/aviator/issues/35' - - link 'related mailing list discussion', - 'https://lists.launchpad.net/openstack/msg24695.html' - - param :all_tenants, :required => false - param :details, :required => false - param :flavor, :required => false - param :image, :required => false - param :limit, :required => false - param :marker, :required => false - param :server, :required => false - param :status, :required => false - param 'changes-since', :required => false, :alias => :changes_since - - - def headers - super - end - - - def http_method - :get - end - - - def url - str = "#{ base_url }/servers" - str += "/detail" if params[:details] - - filters = [] - - (optional_params + required_params - [:details]).each do |param_name| - value = param_name == :all_tenants && params[param_name] ? 1 : params[param_name] - filters << "#{ param_name }=#{ value }" if value - end - - str += "?#{ filters.join('&') }" unless filters.empty? - - str - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/pause_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/pause_server.rb deleted file mode 100644 index 00e5ec663..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/pause_server.rb +++ /dev/null @@ -1,34 +0,0 @@ -module Aviator - - define_request :pause_server, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/POST_pause_v2__tenant_id__servers__server_id__action_ext-os-admin-actions.html' - - param :id, :required => true - - - def body - { :pause => nil } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/reboot_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/reboot_server.rb deleted file mode 100644 index 8aa2296da..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/reboot_server.rb +++ /dev/null @@ -1,41 +0,0 @@ -module Aviator - - define_request :reboot_server, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Reboot_Server-d1e3371.html' - - param :id, :required => true - param :type, :required => false - - - def body - p = { - :reboot => { - :type => params[:type] || 'SOFT' - } - } - - p - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/rebuild_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/rebuild_server.rb deleted file mode 100644 index 8383f571a..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/rebuild_server.rb +++ /dev/null @@ -1,53 +0,0 @@ -module Aviator - - define_request :rebuild_server, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Rebuild_Server-d1e3538.html' - - param :accessIPv4, :required => false, :alias => :access_ipv4 - param :accessIPv6, :required => false, :alias => :access_ipv6 - param :adminPass, :required => true, :alias => :admin_pass - param :id, :required => true - param :imageRef, :required => true, :alias => :image_ref - param :metadata, :required => false - param :name, :required => true - param :personality, :required => false - - - def body - p = { - :rebuild => { - :adminPass => params[:adminPass], - :imageRef => params[:imageRef], - :name => params[:name] - } - } - - [:accessIPv4, :accessIPv6, :metadata, :personality].each do |key| - p[:rebuild][key] = params[key] if params[key] - end - - p - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/resume_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/resume_server.rb deleted file mode 100644 index f220b366c..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/resume_server.rb +++ /dev/null @@ -1,34 +0,0 @@ -module Aviator - - define_request :resume_server, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/POST_resume_v2__tenant_id__servers__server_id__action_ext-os-admin-actions.html' - - param :id, :required => true - - - def body - { :resume => nil } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/root.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/root.rb deleted file mode 100644 index 3a3c4dabc..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/root.rb +++ /dev/null @@ -1,24 +0,0 @@ -module Aviator - - define_request :root, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - def headers - super - end - - - def http_method - :get - end - - - def url - uri = URI(base_url) - "#{ uri.scheme }://#{ uri.host }:#{ uri.port.to_s }/v2/" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/set_image_metadata.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/set_image_metadata.rb deleted file mode 100644 index c17976f9e..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/set_image_metadata.rb +++ /dev/null @@ -1,38 +0,0 @@ -module Aviator - - define_request :set_image_metadata, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Create_or_Replace_Metadata-d1e5358.html' - - - param :id, :required => true - param :metadata, :required => true - - - def body - { - :metadata => params[:metadata] - } - end - - - def headers - super - end - - - def http_method - :put - end - - - def url - "#{ base_url }/images/#{ params[:id] }/metadata" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/set_server_metadata.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/set_server_metadata.rb deleted file mode 100644 index 8631c0397..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/set_server_metadata.rb +++ /dev/null @@ -1,38 +0,0 @@ -module Aviator - - define_request :set_server_metadata, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Create_or_Replace_Metadata-d1e5358.html' - - - param :id, :required => true - param :metadata, :required => true - - - def body - { - :metadata => params[:metadata] - } - end - - - def headers - super - end - - - def http_method - :put - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/metadata" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/suspend_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/suspend_server.rb deleted file mode 100644 index a9bb9a52e..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/suspend_server.rb +++ /dev/null @@ -1,34 +0,0 @@ -module Aviator - - define_request :suspend_server, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/POST_suspend_v2__tenant_id__servers__server_id__action_ext-os-admin-actions.html' - - param :id, :required => true - - - def body - { :suspend => nil } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/unpause_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/unpause_server.rb deleted file mode 100644 index e11089a62..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/unpause_server.rb +++ /dev/null @@ -1,34 +0,0 @@ -module Aviator - - define_request :unpause_server, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/POST_unpause_v2__tenant_id__servers__server_id__action_ext-os-admin-actions.html' - - param :id, :required => true - - - def body - { :unpause => nil } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/action" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_image_metadata.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_image_metadata.rb deleted file mode 100644 index 0ec3694b0..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_image_metadata.rb +++ /dev/null @@ -1,38 +0,0 @@ -module Aviator - - define_request :update_image_metadata, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Update_Metadata-d1e5208.html' - - - param :id, :required => true - param :metadata, :required => true - - - def body - { - :metadata => params[:metadata] - } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/images/#{ params[:id] }/metadata" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_server.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_server.rb deleted file mode 100644 index 453c58329..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_server.rb +++ /dev/null @@ -1,45 +0,0 @@ -module Aviator - - define_request :update_server, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/ServerUpdate.html' - - param :accessIPv4, :required => false, :alias => :access_ipv4 - param :accessIPv6, :required => false, :alias => :access_ipv6 - param :id, :required => true - param :name, :required => false - - - def body - p = { - :server => { } - } - - [:name, :accessIPv4, :accessIPv6].each do |key| - p[:server][key] = params[key] if params[key] - end - - p - end - - - def headers - super - end - - - def http_method - :put - end - - - def url - "#{ base_url }/servers/#{ params[:id] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_server_metadata.rb b/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_server_metadata.rb deleted file mode 100644 index e1e5ad17d..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/compute/requests/v2/public/update_server_metadata.rb +++ /dev/null @@ -1,38 +0,0 @@ -module Aviator - - define_request :update_server_metadata, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :compute - - link 'documentation', - 'http://docs.openstack.org/api/openstack-compute/2/content/Update_Metadata-d1e5208.html' - - - param :id, :required => true - param :metadata, :required => true - - - def body - { - :metadata => params[:metadata] - } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/servers/#{ params[:id] }/metadata" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/add_role_to_user_on_tenant.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/add_role_to_user_on_tenant.rb deleted file mode 100644 index e69b196eb..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/add_role_to_user_on_tenant.rb +++ /dev/null @@ -1,33 +0,0 @@ -module Aviator - - define_request :add_role_to_user_on_tenant, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :identity - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/PUT_addRolesToUserOnTenant_v2.0_tenants__tenantId__users__userId__roles_OS-KSADM__roleId__.html' - - - param :tenant_id, :required => true - param :user_id, :required => true - param :role_id, :required => true - - - def headers - super - end - - - def http_method - :put - end - - - def url - p = params - "#{ base_url }/tenants/#{ p[:tenant_id] }/users/#{ p[:user_id] }/roles/OS-KSADM/#{ p[:role_id] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/create_tenant.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/create_tenant.rb deleted file mode 100644 index 6578ae060..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/create_tenant.rb +++ /dev/null @@ -1,43 +0,0 @@ -module Aviator - - define_request :create_tenant, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :identity - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/' - - - param :name, :required => true - param :description, :required => true - param :enabled, :required => true - - - def body - { - :tenant => { - :name => params[:name], - :description => params[:description], - :enabled => params[:enabled] - } - } - end - - - def headers - super - end - - - def http_method - :post - end - - - def url - "#{ base_url }/tenants" - end - - end - -end \ No newline at end of file diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/create_user.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/create_user.rb deleted file mode 100644 index bf82b64e4..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/create_user.rb +++ /dev/null @@ -1,67 +0,0 @@ -module Aviator - - define_request :create_user do - - meta :provider, :openstack - meta :service, :identity - meta :api_version, :v2 - meta :endpoint_type, :admin - - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/POST_addUser_v2.0_users_.html' - - link 'documentation bug', - 'https://bugs.launchpad.net/keystone/+bug/1110435' - - link 'documentation bug', - 'https://bugs.launchpad.net/keystone/+bug/1226466' - - - param :name, :required => true - param :password, :required => true - - param :email, :required => false - param :enabled, :required => false - param :tenantId, :required => false, :alias => :tenant_id - param :extras, :required => false - - - def body - p = { - :user => {} - } - - (required_params + optional_params).each do |key| - p[:user][key] = params[key] if params[key] - end - - p - end - - - def headers - h = {} - - unless self.anonymous? - h['X-Auth-Token'] = session_data[:body][:access][:token][:id] - end - - h - end - - - def http_method - :post - end - - - def url - service_spec = session_data[:body][:access][:serviceCatalog].find{|s| s[:type] == 'identity' } - "#{ service_spec[:endpoints][0][:adminURL] }/users" - end - - end - -end - diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_role_from_user_on_tenant.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_role_from_user_on_tenant.rb deleted file mode 100644 index 4d24469a7..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_role_from_user_on_tenant.rb +++ /dev/null @@ -1,34 +0,0 @@ -module Aviator - - define_request :delete_role_from_user_on_tenant, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :identity - - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/DELETE_deleteRoleFromUserOnTenant_v2.0_tenants__tenantId__users__userId__roles_OS-KSADM__roleId__.html' - - - param :tenant_id, :required => true - param :user_id, :required => true - param :role_id, :required => true - - - def headers - super - end - - - def http_method - :delete - end - - - def url - p = params - "#{ base_url }/tenants/#{ p[:tenant_id] }/users/#{ p[:user_id] }/roles/OS-KSADM/#{ p[:role_id] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_tenant.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_tenant.rb deleted file mode 100644 index 63ad6437f..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_tenant.rb +++ /dev/null @@ -1,29 +0,0 @@ -module Aviator - - define_request :delete_tenant, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :identity - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/DELETE_deleteTenant_v2.0_tenants__tenantId__.html' - - param :id, :required => true - - - def headers - super - end - - - def http_method - :delete - end - - - def url - "#{ base_url }/tenants/#{ params[:id]}" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_user.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_user.rb deleted file mode 100644 index 0bee46951..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/delete_user.rb +++ /dev/null @@ -1,39 +0,0 @@ -module Aviator - - define_request :delete_user do - - meta :provider, :openstack - meta :service, :identity - meta :api_version, :v2 - meta :endpoint_type, :admin - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/DELETE_deleteUser_v2.0_users__userId__.html' - - param :id, :required => true - - - def headers - h = {} - - unless self.anonymous? - h['X-Auth-Token'] = session_data[:body][:access][:token][:id] - end - - h - end - - - def http_method - :delete - end - - - def url - service_spec = session_data[:body][:access][:serviceCatalog].find{|s| s[:type] == service.to_s } - "#{ service_spec[:endpoints][0][:adminURL] }/users/#{ params[:id]}" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/get_tenant_by_id.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/get_tenant_by_id.rb deleted file mode 100644 index fe2dcc986..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/get_tenant_by_id.rb +++ /dev/null @@ -1,30 +0,0 @@ -module Aviator - - define_request :get_tenant_by_id, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :identity - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/GET_listUsers_v2.0_users_.html' - - - param :id, :required => true - - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/tenants/#{ params[:id] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/get_user.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/get_user.rb deleted file mode 100644 index 9a1e1431a..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/get_user.rb +++ /dev/null @@ -1,33 +0,0 @@ -require 'cgi' - -module Aviator - - define_request :get_user, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :provider, :openstack - meta :service, :identity - meta :api_version, :v2 - meta :endpoint_type, :admin - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/GET_getUserByName_v2.0_users__User_Operations.html' - - param :name, :required => true - - def headers - super - end - - - def http_method - :get - end - - - def url - "#{ base_url }/users?name=#{ CGI::escape(params.name) }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/list_tenants.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/list_tenants.rb deleted file mode 100644 index 48d293062..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/list_tenants.rb +++ /dev/null @@ -1,35 +0,0 @@ -module Aviator - - define_request :list_tenants, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :identity - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/GET_listTenants_v2.0_tenants_Tenant_Operations.html' - - link 'documentation bug', - 'https://bugs.launchpad.net/keystone/+bug/1218601' - - - param :marker, :required => false - param :limit, :required => false - - - def headers - super - end - - - def http_method - :get - end - - - def url - str = "#{ base_url }/tenants" - str += params_to_querystring(optional_params + required_params) - end - - end - -end \ No newline at end of file diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/list_users.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/list_users.rb deleted file mode 100644 index 2debeddc6..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/list_users.rb +++ /dev/null @@ -1,37 +0,0 @@ -module Aviator - - define_request :list_users do - - meta :provider, :openstack - meta :service, :identity - meta :api_version, :v2 - meta :endpoint_type, :admin - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/GET_listUsers_v2.0_users_.html' - - - def headers - h = {} - - unless self.anonymous? - h['X-Auth-Token'] = session_data[:body][:access][:token][:id] - end - - h - end - - - def http_method - :get - end - - - def url - service_spec = session_data[:body][:access][:serviceCatalog].find{|s| s[:type] == 'identity' } - "#{ service_spec[:endpoints][0][:adminURL] }/users" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/update_tenant.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/update_tenant.rb deleted file mode 100644 index 91ae3573d..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/update_tenant.rb +++ /dev/null @@ -1,47 +0,0 @@ -module Aviator - - define_request :update_tenant, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :identity - - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/POST_updateTenant_v2.0_tenants__tenantId__.html' - - - param :id, :required => true - param :name, :required => false - param :enabled, :required => false - param :description, :required => false - - - def body - p = { - :tenant => {} - } - - [:name, :enabled, :description].each do |key| - p[:tenant][key] = params[key] unless params[key].nil? - end - - p - end - - - def headers - super - end - - - def http_method - :put - end - - - def url - "#{ base_url }/tenants/#{ params[:id] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/update_user.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/update_user.rb deleted file mode 100644 index 31f8fb213..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/admin/update_user.rb +++ /dev/null @@ -1,62 +0,0 @@ -module Aviator - - define_request :update_user do - - meta :provider, :openstack - meta :service, :identity - meta :api_version, :v2 - meta :endpoint_type, :admin - - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/POST_updateUser_v2.0_users__userId__.html' - - link 'bug', - 'https://bugs.launchpad.net/keystone/+bug/1226475' - - param :id, :required => true - param :name, :required => false - param :password, :required => false - param :email, :required => false - param :enabled, :required => false - param :tenantId, :required => false, :alias => :tenant_id - param :extras, :required => false - - - def body - p = { - :user => {} - } - - optional_params.each do |key| - p[:user][key] = params[key] if params[key] - end - - p - end - - - def headers - h = {} - - unless self.anonymous? - h['X-Auth-Token'] = session_data[:body][:access][:token][:id] - end - - h - end - - - def http_method - :put - end - - - def url - service_spec = session_data[:body][:access][:serviceCatalog].find { |s| s[:type] == service.to_s } - "#{ service_spec[:endpoints][0][:adminURL] }/users/#{ params[:id] }" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/create_token.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/create_token.rb deleted file mode 100644 index 91731d462..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/create_token.rb +++ /dev/null @@ -1,62 +0,0 @@ -module Aviator - - define_request :create_token, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :anonymous, true - meta :service, :identity - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/POST_authenticate_v2.0_tokens_.html' - - link 'documentation bug', - 'https://bugs.launchpad.net/keystone/+bug/1208607' - - - param :username, :required => false - param :password, :required => false - param :tokenId, :required => false, :alias => :token_id - param :tenantName, :required => false, :alias => :tenant_name - param :tenantId, :required => false, :alias => :tenant_id - - - def body - p = if params[:tokenId] - { - :auth => { - :token => { - :id => params[:tokenId] - } - } - } - else - { - :auth => { - :passwordCredentials => { - :username => params[:username], - :password => params[:password] - } - } - } - end - - p[:auth][:tenantName] = params[:tenantName] if params[:tenantName] - p[:auth][:tenantId] = params[:tenantId] if params[:tenantId] - - p - end - - - def http_method - :post - end - - - def url - url = session_data[:auth_service][:host_uri] - url += '/v2.0' if (URI(url).path =~ /^\/?\w+/).nil? - url += "/tokens" - end - - end - -end \ No newline at end of file diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/list_tenants.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/list_tenants.rb deleted file mode 100644 index 094032ab0..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/list_tenants.rb +++ /dev/null @@ -1,35 +0,0 @@ -module Aviator - - define_request :list_tenants, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :identity - - link 'documentation', - 'http://docs.openstack.org/api/openstack-identity-service/2.0/content/GET_listTenants_v2.0_tokens_tenants_.html' - - link 'documentation bug', - 'https://bugs.launchpad.net/keystone/+bug/1218601' - - - param :marker, :required => false - param :limit, :required => false - - - def url - str = "#{ base_url }/tenants" - str += params_to_querystring(optional_params + required_params) - end - - - def headers - super - end - - - def http_method - :get - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/root.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/root.rb deleted file mode 100644 index 0c6d38358..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v2/public/root.rb +++ /dev/null @@ -1,25 +0,0 @@ -module Aviator - - define_request :root, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :identity - - - def headers - super - end - - - def http_method - :get - end - - - def url - uri = URI(base_url) - "#{ uri.scheme }://#{ uri.host }:#{ uri.port.to_s }/v2.0/" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v3/public/create_token.rb b/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v3/public/create_token.rb deleted file mode 100644 index 17d8bb078..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/identity/requests/v3/public/create_token.rb +++ /dev/null @@ -1,125 +0,0 @@ -module Aviator - # Original work by Stephen Paul Suarez - # https://github.com/musashi-dev/aviator/blob/develop/lib/aviator/openstack/identity/v3/public/create_token.rb - - define_request :create_token, :inherit => [:openstack, :common, :v3, :public, :base] do - - meta :anonymous, true - meta :service, :identity - meta :api_version, :v3 - - link 'documentation', - 'http://api.openstack.org/api-ref-identity-v3.html#Token_Calls' - - param :domainId, :required => false, :alias => :domain_id - param :domainName, :required => false, :alias => :domain_name - param :password, :required => false - param :tenantId, :required => false, :alias => :tenant_id - param :tenantName, :required => false, :alias => :tenant_name - param :tokenId, :required => false, :alias => :token_id - param :userId, :required => false, :alias => :user_id - param :username, :required => false - - - def body - params[:token_id] ? token_auth_body : password_auth_body - end - - - def http_method - :post - end - - - def url - url = session_data[:auth_service][:host_uri] - url += '/v3' if (URI(url).path =~ /^\/?\w+/).nil? - url += "/auth/tokens" - end - - - private - - # Removes nil elements from hash - # Adapted from http://stackoverflow.com/a/14773555/402145 - def compact_hash(hash, opts = {}) - opts[:recurse] ||= true - hash.inject({}) do |new_hash, (k,v)| - if !v.nil? - new_hash[k] = opts[:recurse] && v.kind_of?(Hash) ? compact_hash(v, opts) : v - end - new_hash - end - end - - - def domain_hash - compact_hash({ - :id => params[:domain_id], - :name => params[:domain_name] - }) - end - - - def password_auth_body - p = { - :auth => { - :identity => { - :methods => ['password'], - :password => { - :user => compact_hash({ - :id => params[:user_id], - :name => params[:username], - :password => params[:password] - }) - } - } - } - } - - if params[:domain_name] || params[:domain_id] - p[:auth][:identity][:password][:user][:domain] = domain_hash - end - - if params[:tenant_name] || params[:tenant_id] || params[:domain_name] || params[:domain_id] - p[:auth][:scope] = scope_hash - end - p - end - - - def scope_hash - p = {} - - if params[:tenant_name] || params[:tenant_id] - p[:project] = compact_hash({ - :id => params[:tenant_id], - :name => params[:tenant_name] - }) - p[:project][:domain] = domain_hash if params[:domain_name] || params[:domain_id] - - elsif params[:domain_name] || params[:domain_id] - p[:domain] = domain_hash - end - - p - end - - - def token_auth_body - p = { - :auth => { - :identity => { - :methods => ['token'], - :token => { :id => params[:token_id] } - } - } - } - p[:auth][:scope] = scope_hash if params[:tenant_name] || params[:tenant_id] - p - end - - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/image/requests/v1/public/list_public_images.rb b/aviator/lib/puppet/feature/aviator/openstack/image/requests/v1/public/list_public_images.rb deleted file mode 100644 index 2164d73eb..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/image/requests/v1/public/list_public_images.rb +++ /dev/null @@ -1,45 +0,0 @@ -module Aviator - - define_request :list_public_images, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :image - meta :api_version, :v1 - - link 'documentation', 'http://docs.openstack.org/api/openstack-image-service/1.1/content/requesting-a-list-of-public-vm-images.html' - - param :name, :required => false - param :container_format, :required => false - param :disk_format, :required => false - param :status, :required => false - param :size_min, :required => false - param :size_max, :required => false - param :sort_key, :required => false - param :sort_dir, :required => false - - - def headers - super - end - - def http_method - :get - end - - def url - uri = URI(base_url) - url = "#{ uri.scheme }://#{ uri.host }:#{ uri.port.to_s }/v1/images" - - filters = [] - - optional_params.each do |param_name| - filters << "#{ param_name }=#{ params[param_name] }" if params[param_name] - end - - url += "?#{ filters.join('&') }" unless filters.empty? - - url - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/image/requests/v1/public/root.rb b/aviator/lib/puppet/feature/aviator/openstack/image/requests/v1/public/root.rb deleted file mode 100644 index 553f5369e..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/image/requests/v1/public/root.rb +++ /dev/null @@ -1,25 +0,0 @@ -module Aviator - - define_request :root, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :image - meta :api_version, :v1 - - def headers - super - end - - - def http_method - :get - end - - - def url - uri = URI(base_url) - "#{ uri.scheme }://#{ uri.host }:#{ uri.port.to_s }/v1/" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/metering/requests/v1/admin/list_projects.rb b/aviator/lib/puppet/feature/aviator/openstack/metering/requests/v1/admin/list_projects.rb deleted file mode 100644 index 7ec8e667d..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/metering/requests/v1/admin/list_projects.rb +++ /dev/null @@ -1,27 +0,0 @@ -module Aviator - - define_request :list_projects, :inherit => [:openstack, :common, :v2, :admin, :base] do - - meta :service, :metering - meta :api_version, :v1 - meta :endpoint_type, :admin - - - def headers - super - end - - - def http_method - :get - end - - - def url - uri = URI(base_url) - "#{ uri.scheme }://#{ uri.host }:#{ uri.port.to_s }/v1/projects" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/provider.rb b/aviator/lib/puppet/feature/aviator/openstack/provider.rb deleted file mode 100644 index a4c64e806..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/provider.rb +++ /dev/null @@ -1,187 +0,0 @@ -module Aviator -module Openstack - - # - # Request Options - # - # The following options may be used in combination with each other when calling - # an OpenStack request class - # - # :api_version => :v2:: - # Forces Aviator to use the request class for the v2 API. For any other - # version, replace :v2 with the desired one. Note that this may throw an - # error if no such request class for the given api version exists. If you - # want to globally specify the API version to use for a specific service, - # declare it in your config file under the correct environment. For example: - # - # production: - # provider: openstack - # ... - # compute_service: - # api_version: v2 - # - # Note that the :api_version option overrides whatever is declared in the - # configuration. - # - # :endpoint_type => (:public|:admin):: - # This allows you to be specific about the endpoint type in cases where two - # request classes under admin and public endpoints of the same service share - # the same name. This is true, for example, for the :list_tenants request of - # the identity service's v2 API. Its public endpoint will return only the tenants - # the user is a member of whereas the admin endpoint will return all tenants - # in the system. - # - # :session_data => Hash:: - # Under normal situations, you wouldn't need to use this as it is automatically populated - # by the Session object provided it is authenticated. The specific use case when you'd - # need to set thsi optin is when you want to use Aviator to seed your OpenStack installation. - # In such a scenario, you would need to use a service token since no usernames and tenants - # would exist yet in the environment. To use a service token with Aviator, you will need to - # write something similar to the following example: - # - # openstack = Aviator::Session.new(:config => { :provider => 'openstack'}) - # - # session_data = {:base_url => 'http://example.com', - # :service_token => 'service-token-created-at-openstack-install-time'} - # - # openstack.request :identity, :create_tenant, :api_version => :v2, :session_data => session_data) do |params| - # params.name = 'Tenant A' - # params.description = 'First Tenant!' - # params.enabled = true - # end - # - # Notice how the above code skips authentication. This is because the service token is - # pre-validated and ready for use with any request. Also note how we're providing a :base_url - # member in our session data. This is necessary since we normally get the service endpoints from - # Keystone when we authenticate. Now since we are not authenticating against Keystone, we don't have - # that catalogue to begin with. Thus the need to hardcode it in the request. - # - module Provider - - class MultipleServiceApisError < StandardError - def initialize(service, entries, request_name) - types = entries.map{|e| e[:type] }.join("\n - ") - msg = < :v2 { ... } - -If you combine the two methods, method #2 will override method #1 - -EOF - super(msg) - end - end - - class << self - - #:nodoc: - def find_request(service, name, session_data, options) - service = service.to_s - endpoint_type = options[:endpoint_type] - endpoint_types = if endpoint_type - [endpoint_type.to_s.camelize] - else - ['Public', 'Admin'] - end - - namespace = Aviator.const_get('Openstack') \ - .const_get(service.camelize) \ - .const_get('Requests') - - if options[:api_version] - m = options[:api_version].to_s.match(/(v\d+)\.?\d*/) - version = m[1].to_s.camelize unless m.nil? - end - - version ||= infer_version(session_data, name, service) - - unless version.nil? - version = version.to_s.camelize - end - - return nil unless version && namespace.const_defined?(version) - - namespace = namespace.const_get(version, name) - - endpoint_types.each do |endpoint_type| - name = name.to_s.camelize - - next unless namespace.const_defined?(endpoint_type) - next unless namespace.const_get(endpoint_type).const_defined?(name) - - return namespace.const_get(endpoint_type).const_get(name) - end - - nil - end - - - def root_dir - Pathname.new(__FILE__).join('..').expand_path - end - - - def request_file_paths(service) - Dir.glob(Pathname.new(__FILE__).join( - '..', - service.to_s, - 'requests', - '**', - '*.rb' - ).expand_path - ) - end - - - private - - def infer_version(session_data, request_name, service) - if session_data.has_key?(:auth_service) && session_data[:auth_service][:api_version] - session_data[:auth_service][:api_version].to_sym - - elsif session_data.has_key?(:auth_service) && session_data[:auth_service][:host_uri] - m = session_data[:auth_service][:host_uri].match(/(v\d+)\.?\d*/) - return m[1].to_sym unless m.nil? - - elsif session_data.has_key? :base_url - m = session_data[:base_url].match(/(v\d+)\.?\d*/) - return m[1].to_sym unless m.nil? - - elsif session_data.has_key?(:body) && session_data[:body].has_key?(:access) - service_specs = session_data[:body][:access][:serviceCatalog].select{|s| s[:type].match("#{ service }(v\d+)?") } - raise MultipleServiceApisError.new(service, service_specs, request_name) unless service_specs.length <= 1 - raise Aviator::Service::MissingServiceEndpointError.new(service.to_s, request_name) unless service_specs.length > 0 - version = service_specs[0][:endpoints][0][:publicURL].match(/(v\d+)\.?\d*/) - version ? version[1].to_sym : :v1 - - elsif session_data.has_key?(:headers) && session_data[:headers].has_key?("x-subject-token") - service_specs = session_data[:body][:token][:catalog].select{|s| s[:type].match("#{ service }(v\d+)?") } - raise MultipleServiceApisError.new(service, service_specs, request_name) unless service_specs.length <= 1 - raise Aviator::Service::MissingServiceEndpointError.new(service.to_s, request_name) unless service_specs.length > 0 - version = service_specs[0][:endpoints][0][:url].match(/(v\d+)\.?\d*/) - version ? version[1].to_sym : :v1 - end - end - - end # class << self - - end # module Provider - -end # module Openstack -end # module Aviator diff --git a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/create_volume.rb b/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/create_volume.rb deleted file mode 100644 index f40354d99..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/create_volume.rb +++ /dev/null @@ -1,47 +0,0 @@ -module Aviator - - define_request :create_volume, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :volume - meta :api_version, :v1 - - link 'documentation', 'http://docs.rackspace.com/cbs/api/v1.0/cbs-devguide/content/POST_createVolume_v1__tenant_id__volumes_v1__tenant_id__volumes.html' - - param :display_name, :required => true - param :display_description, :required => true - param :size, :required => true - param :volume_type, :required => false - param :availability_zone, :required => false - param :snapshot_id, :required => false - param :metadata, :required => false - - def body - p = { - :volume => { - :display_name => params[:display_name], - :display_description => params[:display_description], - :size => params[:size] - } - } - - optional_params.each do |key| - p[:volume][key] = params[key] if params[key] - end - - p - end - - def headers - super - end - - def http_method - :post - end - - def url - "#{ base_url }/volumes" - end - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/delete_volume.rb b/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/delete_volume.rb deleted file mode 100644 index 1397959aa..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/delete_volume.rb +++ /dev/null @@ -1,25 +0,0 @@ -module Aviator - - define_request :delete_volume, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :volume - meta :api_version, :v1 - - link 'documentation', 'http://docs.rackspace.com/cbs/api/v1.0/cbs-devguide/content/DELETE_deleteVolume_v1__tenant_id__volumes__volume_id__v1__tenant_id__volumes.html' - - param :id, :required => true - - def headers - super - end - - def http_method - :delete - end - - def url - "#{ base_url }/volumes/#{ params[:id] }" - end - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/get_volume.rb b/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/get_volume.rb deleted file mode 100644 index 991dc4454..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/get_volume.rb +++ /dev/null @@ -1,28 +0,0 @@ -module Aviator - - define_request :get_volume, :inherit => [:openstack, :common, :v2, :public, :base] do - meta :provider, :openstack - meta :service, :volume - meta :api_version, :v1 - meta :endpoint_type, :public - - link 'documentation', 'http://docs.rackspace.com/cbs/api/v1.0/cbs-devguide/content/GET_getVolume_v1__tenant_id__volumes__volume_id__v1__tenant_id__volumes.html' - - param :id, :required => true - - def headers - super - end - - def http_method - :get - end - - def url - "#{ base_url }/volumes/#{ params[:id] }" - end - - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/list_volume_types.rb b/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/list_volume_types.rb deleted file mode 100644 index d2f0b5c8a..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/list_volume_types.rb +++ /dev/null @@ -1,29 +0,0 @@ -module Aviator - - define_request :list_volume_types, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :provider, :openstack - meta :service, :volume - meta :api_version, :v1 - meta :endpoint_type, :public - - link 'documentation', 'http://docs.rackspace.com/cbs/api/v1.0/cbs-devguide/content/GET_getVolumeTypes_v1__tenant_id__types_v1__tenant_id__types.html' - - param :extra_specs, :required => false - param :name, :required => false - - def headers - super - end - - def http_method - :get - end - - def url - "#{ base_url }/types" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/list_volumes.rb b/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/list_volumes.rb deleted file mode 100644 index c88cf7500..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/list_volumes.rb +++ /dev/null @@ -1,48 +0,0 @@ -module Aviator - - define_request :list_volumes, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :volume - meta :api_version, :v1 - - link 'documentation', 'http://docs.rackspace.com/cbs/api/v1.0/cbs-devguide/content/GET_getVolumesSimple_v1__tenant_id__volumes_v1__tenant_id__volumes.html' - - param :all_tenants, :required => false - param :details, :required => false - param :status, :required => false - param :availability_zone, :required => false - param :bootable, :required => false - param :display_name, :required => false - param :display_description, :required => false - param :volume_type, :required => false - param :snapshot_id, :required => false - param :size, :required => false - - - def headers - super - end - - def http_method - :get - end - - def url - str = "#{ base_url }/volumes" - str += "/detail" if params[:details] - - filters = [] - - (optional_params + required_params - [:details]).each do |param_name| - value = param_name == :all_tenants && params[param_name] ? 1 : params[param_name] - filters << "#{ param_name }=#{ value }" if value - end - - str += "?#{ filters.join('&') }" unless filters.empty? - - str - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/root.rb b/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/root.rb deleted file mode 100644 index 0e3748c9c..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/root.rb +++ /dev/null @@ -1,26 +0,0 @@ -module Aviator - - define_request :root, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :volume - meta :api_version, :v1 - - - def headers - super - end - - - def http_method - :get - end - - - def url - uri = URI(base_url) - "#{ uri.scheme }://#{ uri.host }:#{ uri.port.to_s }/v1/" - end - - end - -end diff --git a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/update_volume.rb b/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/update_volume.rb deleted file mode 100644 index 67121c36f..000000000 --- a/aviator/lib/puppet/feature/aviator/openstack/volume/requests/v1/public/update_volume.rb +++ /dev/null @@ -1,43 +0,0 @@ -module Aviator - - define_request :update_volume, :inherit => [:openstack, :common, :v2, :public, :base] do - - meta :service, :volume - meta :api_version, :v1 - - link 'documentation', 'http://docs.rackspace.com/cbs/api/v1.0/cbs-devguide/content/PUT_renameVolume_v1__tenant_id__volumes__volume_id__v1__tenant_id__volumes.html' - - param :id, :required => true - param :display_name, :required => false - param :display_description, :required => false - - - def body - p = { :volume => {} } - - [:display_name, :display_description].each do |key| - p[:volume][key] = params[key] if params[key] - end - - p - end - - - def headers - super - end - - - def http_method - :put - end - - - def url - "#{ base_url }/volumes/#{ params[:id] }" - end - - end - - -end diff --git a/aviator/lib/puppet/feature/aviator/version.rb b/aviator/lib/puppet/feature/aviator/version.rb deleted file mode 100644 index faa1543f9..000000000 --- a/aviator/lib/puppet/feature/aviator/version.rb +++ /dev/null @@ -1,3 +0,0 @@ -module Aviator - VERSION = "0.2.1" -end diff --git a/aviator/lib/puppet/feature/composite_io.rb b/aviator/lib/puppet/feature/composite_io.rb deleted file mode 100644 index 4ba7cf5ff..000000000 --- a/aviator/lib/puppet/feature/composite_io.rb +++ /dev/null @@ -1,108 +0,0 @@ -#-- -# Copyright (c) 2007-2012 Nick Sieger. -# See the file README.txt included with the distribution for -# software license details. -#++ - -# Concatenate together multiple IO objects into a single, composite IO object -# for purposes of reading as a single stream. -# -# Usage: -# -# crio = CompositeReadIO.new(StringIO.new('one'), StringIO.new('two'), StringIO.new('three')) -# puts crio.read # => "onetwothree" -# -class CompositeReadIO - # Create a new composite-read IO from the arguments, all of which should - # respond to #read in a manner consistent with IO. - def initialize(*ios) - @ios = ios.flatten - @index = 0 - end - - # Read from IOs in order until `length` bytes have been received. - def read(length = nil, outbuf = nil) - got_result = false - outbuf = outbuf ? outbuf.replace("") : "" - - while io = current_io - if result = io.read(length) - got_result ||= !result.nil? - result.force_encoding("BINARY") if result.respond_to?(:force_encoding) - outbuf << result - length -= result.length if length - break if length == 0 - end - advance_io - end - (!got_result && length) ? nil : outbuf - end - - def rewind - @ios.each { |io| io.rewind } - @index = 0 - end - - private - - def current_io - @ios[@index] - end - - def advance_io - @index += 1 - end -end - -# Convenience methods for dealing with files and IO that are to be uploaded. -class UploadIO - # Create an upload IO suitable for including in the params hash of a - # Net::HTTP::Post::Multipart. - # - # Can take two forms. The first accepts a filename and content type, and - # opens the file for reading (to be closed by finalizer). - # - # The second accepts an already-open IO, but also requires a third argument, - # the filename from which it was opened (particularly useful/recommended if - # uploading directly from a form in a framework, which often save the file to - # an arbitrarily named RackMultipart file in /tmp). - # - # Usage: - # - # UploadIO.new("file.txt", "text/plain") - # UploadIO.new(file_io, "text/plain", "file.txt") - # - attr_reader :content_type, :original_filename, :local_path, :io, :opts - - def initialize(filename_or_io, content_type, filename = nil, opts = {}) - io = filename_or_io - local_path = "" - if io.respond_to? :read - # in Ruby 1.9.2, StringIOs no longer respond to path - # (since they respond to :length, so we don't need their local path, see parts.rb:41) - local_path = filename_or_io.respond_to?(:path) ? filename_or_io.path : "local.path" - else - io = File.open(filename_or_io) - local_path = filename_or_io - end - filename ||= local_path - - @content_type = content_type - @original_filename = File.basename(filename) - @local_path = local_path - @io = io - @opts = opts - end - - def self.convert!(io, content_type, original_filename, local_path) - raise ArgumentError, "convert! has been removed. You must now wrap IOs using:\nUploadIO.new(filename_or_io, content_type, filename=nil)\nPlease update your code." - end - - def method_missing(*args) - @io.send(*args) - end - - def respond_to?(meth, include_all = false) - @io.respond_to?(meth, include_all) || super(meth, include_all) - end -end diff --git a/aviator/lib/puppet/feature/faraday.rb b/aviator/lib/puppet/feature/faraday.rb deleted file mode 100644 index 4b63ec8e9..000000000 --- a/aviator/lib/puppet/feature/faraday.rb +++ /dev/null @@ -1,268 +0,0 @@ -require 'thread' -require 'cgi' -require 'set' -require 'forwardable' - -# Public: This is the main namespace for Faraday. You can either use it to -# create Faraday::Connection objects, or access it directly. -# -# Examples -# -# Faraday.get "http://faraday.com" -# -# conn = Faraday.new "http://faraday.com" -# conn.get '/' -# -module Faraday - VERSION = "0.9.0" - - class << self - # Public: Gets or sets the root path that Faraday is being loaded from. - # This is the root from where the libraries are auto-loaded from. - attr_accessor :root_path - - # Public: Gets or sets the path that the Faraday libs are loaded from. - attr_accessor :lib_path - - # Public: Gets or sets the Symbol key identifying a default Adapter to use - # for the default Faraday::Connection. - attr_reader :default_adapter - - # Public: Sets the default Faraday::Connection for simple scripts that - # access the Faraday constant directly. - # - # Faraday.get "https://faraday.com" - attr_writer :default_connection - - # Public: Sets the default options used when calling Faraday#new. - attr_writer :default_connection_options - - # Public: Initializes a new Faraday::Connection. - # - # url - The optional String base URL to use as a prefix for all - # requests. Can also be the options Hash. - # options - The optional Hash used to configure this Faraday::Connection. - # Any of these values will be set on every request made, unless - # overridden for a specific request. - # :url - String base URL. - # :params - Hash of URI query unencoded key/value pairs. - # :headers - Hash of unencoded HTTP header key/value pairs. - # :request - Hash of request options. - # :ssl - Hash of SSL options. - # :proxy - Hash of Proxy options. - # - # Examples - # - # Faraday.new 'http://faraday.com' - # - # # http://faraday.com?page=1 - # Faraday.new 'http://faraday.com', :params => {:page => 1} - # - # # same - # - # Faraday.new :url => 'http://faraday.com', - # :params => {:page => 1} - # - # Returns a Faraday::Connection. - def new(url = nil, options = nil) - block = block_given? ? Proc.new : nil - options = options ? default_connection_options.merge(options) : default_connection_options.dup - Faraday::Connection.new(url, options, &block) - end - - # Internal: Requires internal Faraday libraries. - # - # *libs - One or more relative String names to Faraday classes. - # - # Returns nothing. - def require_libs(*libs) - libs.each do |lib| - require "#{lib_path}/#{lib}" - end - end - - # Public: Updates default adapter while resetting - # #default_connection. - # - # Returns the new default_adapter. - def default_adapter=(adapter) - @default_connection = nil - @default_adapter = adapter - end - - alias require_lib require_libs - - private - # Internal: Proxies method calls on the Faraday constant to - # #default_connection. - def method_missing(name, *args, &block) - default_connection.send(name, *args, &block) - end - end - - self.root_path = File.expand_path "..", __FILE__ - self.lib_path = File.expand_path "../faraday", __FILE__ - self.default_adapter = :net_http - - # Gets the default connection used for simple scripts. - # - # Returns a Faraday::Connection, configured with the #default_adapter. - def self.default_connection - @default_connection ||= Connection.new - end - - # Gets the default connection options used when calling Faraday#new. - # - # Returns a Faraday::ConnectionOptions. - def self.default_connection_options - @default_connection_options ||= ConnectionOptions.new - end - - if (!defined?(RUBY_ENGINE) || "ruby" == RUBY_ENGINE) && RUBY_VERSION < '1.9' - begin - require 'system_timer' - Timer = SystemTimer - rescue LoadError - warn "Faraday: you may want to install system_timer for reliable timeouts" - end - end - - unless const_defined? :Timer - require 'timeout' - Timer = Timeout - end - - # Public: Adds the ability for other modules to register and lookup - # middleware classes. - module MiddlewareRegistry - # Public: Register middleware class(es) on the current module. - # - # mapping - A Hash mapping Symbol keys to classes. Classes can be expressed - # as fully qualified constant, or a Proc that will be lazily - # called to return the former. - # - # Examples - # - # module Faraday - # class Whatever - # # Middleware looked up by :foo returns Faraday::Whatever::Foo. - # register_middleware :foo => Foo - # - # # Middleware looked up by :bar returns Faraday::Whatever.const_get(:Bar) - # register_middleware :bar => :Bar - # - # # Middleware looked up by :baz requires 'baz' and returns Faraday::Whatever.const_get(:Baz) - # register_middleware :baz => [:Baz, 'baz'] - # end - # end - # - # Returns nothing. - def register_middleware(autoload_path = nil, mapping = nil) - if mapping.nil? - mapping = autoload_path - autoload_path = nil - end - middleware_mutex do - @middleware_autoload_path = autoload_path if autoload_path - (@registered_middleware ||= {}).update(mapping) - end - end - - # Public: Lookup middleware class with a registered Symbol shortcut. - # - # key - The Symbol key for the registered middleware. - # - # Examples - # - # module Faraday - # class Whatever - # register_middleware :foo => Foo - # end - # end - # - # Faraday::Whatever.lookup_middleware(:foo) - # # => Faraday::Whatever::Foo - # - # Returns a middleware Class. - def lookup_middleware(key) - load_middleware(key) || - raise(Faraday::Error.new("#{key.inspect} is not registered on #{self}")) - end - - def middleware_mutex(&block) - @middleware_mutex ||= begin - require 'monitor' - Monitor.new - end - @middleware_mutex.synchronize(&block) - end - - def fetch_middleware(key) - defined?(@registered_middleware) && @registered_middleware[key] - end - - def load_middleware(key) - value = fetch_middleware(key) - case value - when Module - value - when Symbol, String - middleware_mutex do - @registered_middleware[key] = const_get(value) - end - when Proc - middleware_mutex do - @registered_middleware[key] = value.call - end - when Array - middleware_mutex do - const, path = value - if root = @middleware_autoload_path - path = "#{root}/#{path}" - end - require(path) - @registered_middleware[key] = const - end - load_middleware(key) - end - end - end - - def self.const_missing(name) - if name.to_sym == :Builder - warn "Faraday::Builder is now Faraday::RackBuilder." - const_set name, RackBuilder - else - super - end - end - - require_libs "utils", "options", "connection", "rack_builder", "parameters", - "middleware", "adapter", "request", "response", "upload_io", "error" - - if !ENV["FARADAY_NO_AUTOLOAD"] - require_lib 'autoload' - end -end - -# not pulling in active-support JUST for this method. And I love this method. -class Object - # The primary purpose of this method is to "tap into" a method chain, - # in order to perform operations on intermediate results within the chain. - # - # Examples - # - # (1..10).tap { |x| puts "original: #{x.inspect}" }.to_a. - # tap { |x| puts "array: #{x.inspect}" }. - # select { |x| x%2 == 0 }. - # tap { |x| puts "evens: #{x.inspect}" }. - # map { |x| x*x }. - # tap { |x| puts "squares: #{x.inspect}" } - # - # Yields self. - # Returns self. - def tap - yield(self) - self - end unless Object.respond_to?(:tap) -end diff --git a/aviator/lib/puppet/feature/faraday/adapter.rb b/aviator/lib/puppet/feature/faraday/adapter.rb deleted file mode 100644 index f018b509a..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter.rb +++ /dev/null @@ -1,46 +0,0 @@ -module Faraday - # Public: This is a base class for all Faraday adapters. Adapters are - # responsible for fulfilling a Faraday request. - class Adapter < Middleware - CONTENT_LENGTH = 'Content-Length'.freeze - - register_middleware File.expand_path('../adapter', __FILE__), - :test => [:Test, 'test'], - :net_http => [:NetHttp, 'net_http'], - :net_http_persistent => [:NetHttpPersistent, 'net_http_persistent'], - :typhoeus => [:Typhoeus, 'typhoeus'], - :patron => [:Patron, 'patron'], - :em_synchrony => [:EMSynchrony, 'em_synchrony'], - :em_http => [:EMHttp, 'em_http'], - :excon => [:Excon, 'excon'], - :rack => [:Rack, 'rack'], - :httpclient => [:HTTPClient, 'httpclient'] - - # Public: This module marks an Adapter as supporting parallel requests. - module Parallelism - attr_writer :supports_parallel - def supports_parallel?() @supports_parallel end - - def inherited(subclass) - super - subclass.supports_parallel = self.supports_parallel? - end - end - - extend Parallelism - self.supports_parallel = false - - def call(env) - env.clear_body if env.needs_body? - end - - def save_response(env, status, body, headers = nil) - env.status = status - env.body = body - env.response_headers = Utils::Headers.new.tap do |response_headers| - response_headers.update headers unless headers.nil? - yield(response_headers) if block_given? - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/adapter/em_http.rb b/aviator/lib/puppet/feature/faraday/adapter/em_http.rb deleted file mode 100644 index a248fcfd2..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/em_http.rb +++ /dev/null @@ -1,237 +0,0 @@ -module Faraday - class Adapter - # EventMachine adapter is useful for either asynchronous requests - # when in EM reactor loop or for making parallel requests in - # synchronous code. - class EMHttp < Faraday::Adapter - module Options - def connection_config(env) - options = {} - configure_proxy(options, env) - configure_timeout(options, env) - configure_socket(options, env) - configure_ssl(options, env) - options - end - - def request_config(env) - options = { - :body => read_body(env), - :head => env[:request_headers], - # :keepalive => true, - # :file => 'path/to/file', # stream data off disk - } - configure_compression(options, env) - options - end - - def read_body(env) - body = env[:body] - body.respond_to?(:read) ? body.read : body - end - - def configure_proxy(options, env) - if proxy = request_options(env)[:proxy] - options[:proxy] = { - :host => proxy[:uri].host, - :port => proxy[:uri].port, - :authorization => [proxy[:user], proxy[:password]] - } - end - end - - def configure_socket(options, env) - if bind = request_options(env)[:bind] - options[:bind] = { - :host => bind[:host], - :port => bind[:port] - } - end - end - - def configure_ssl(options, env) - if env[:url].scheme == 'https' && env[:ssl] - options[:ssl] = { - :cert_chain_file => env[:ssl][:ca_file], - :verify_peer => env[:ssl].fetch(:verify, true) - } - end - end - - def configure_timeout(options, env) - timeout, open_timeout = request_options(env).values_at(:timeout, :open_timeout) - options[:connect_timeout] = options[:inactivity_timeout] = timeout - options[:connect_timeout] = open_timeout if open_timeout - end - - def configure_compression(options, env) - if env[:method] == :get and not options[:head].key? 'accept-encoding' - options[:head]['accept-encoding'] = 'gzip, compressed' - end - end - - def request_options(env) - env[:request] - end - end - - include Options - - dependency 'em-http' - - self.supports_parallel = true - - def self.setup_parallel_manager(options = nil) - Manager.new - end - - def call(env) - super - perform_request env - @app.call env - end - - def perform_request(env) - if parallel?(env) - manager = env[:parallel_manager] - manager.add { - perform_single_request(env). - callback { env[:response].finish(env) } - } - else - unless EventMachine.reactor_running? - error = nil - # start EM, block until request is completed - EventMachine.run do - perform_single_request(env). - callback { EventMachine.stop }. - errback { |client| - error = error_message(client) - EventMachine.stop - } - end - raise_error(error) if error - else - # EM is running: instruct upstream that this is an async request - env[:parallel_manager] = true - perform_single_request(env). - callback { env[:response].finish(env) }. - errback { - # TODO: no way to communicate the error in async mode - raise NotImplementedError - } - end - end - rescue EventMachine::Connectify::CONNECTError => err - if err.message.include?("Proxy Authentication Required") - raise Error::ConnectionFailed, %{407 "Proxy Authentication Required "} - else - raise Error::ConnectionFailed, err - end - rescue => err - if defined?(OpenSSL) && OpenSSL::SSL::SSLError === err - raise Faraday::SSLError, err - else - raise - end - end - - # TODO: reuse the connection to support pipelining - def perform_single_request(env) - req = EventMachine::HttpRequest.new(env[:url], connection_config(env)) - req.setup_request(env[:method], request_config(env)).callback { |client| - save_response(env, client.response_header.status, client.response) do |resp_headers| - client.response_header.each do |name, value| - resp_headers[name.to_sym] = value - end - end - } - end - - def error_message(client) - client.error or "request failed" - end - - def raise_error(msg) - errklass = Faraday::Error::ClientError - if msg == Errno::ETIMEDOUT - errklass = Faraday::Error::TimeoutError - msg = "request timed out" - elsif msg == Errno::ECONNREFUSED - errklass = Faraday::Error::ConnectionFailed - msg = "connection refused" - elsif msg == "connection closed by server" - errklass = Faraday::Error::ConnectionFailed - end - raise errklass, msg - end - - def parallel?(env) - !!env[:parallel_manager] - end - - # The parallel manager is designed to start an EventMachine loop - # and block until all registered requests have been completed. - class Manager - def initialize - reset - end - - def reset - @registered_procs = [] - @num_registered = 0 - @num_succeeded = 0 - @errors = [] - @running = false - end - - def running?() @running end - - def add - if running? - perform_request { yield } - else - @registered_procs << Proc.new - end - @num_registered += 1 - end - - def run - if @num_registered > 0 - @running = true - EventMachine.run do - @registered_procs.each do |proc| - perform_request(&proc) - end - end - if @errors.size > 0 - raise Faraday::Error::ClientError, @errors.first || "connection failed" - end - end - ensure - reset - end - - def perform_request - client = yield - client.callback { @num_succeeded += 1; check_finished } - client.errback { @errors << client.error; check_finished } - end - - def check_finished - if @num_succeeded + @errors.size == @num_registered - EventMachine.stop - end - end - end - end - end -end - -begin - require 'openssl' -rescue LoadError - warn "Warning: no such file to load -- openssl. Make sure it is installed if you want HTTPS support" -else - require 'faraday/adapter/em_http_ssl_patch' -end if Faraday::Adapter::EMHttp.loaded? diff --git a/aviator/lib/puppet/feature/faraday/adapter/em_http_ssl_patch.rb b/aviator/lib/puppet/feature/faraday/adapter/em_http_ssl_patch.rb deleted file mode 100644 index 8bbfcbce3..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/em_http_ssl_patch.rb +++ /dev/null @@ -1,56 +0,0 @@ -require 'openssl' -require 'em-http' - -module EmHttpSslPatch - def ssl_verify_peer(cert_string) - cert = nil - begin - cert = OpenSSL::X509::Certificate.new(cert_string) - rescue OpenSSL::X509::CertificateError - return false - end - - @last_seen_cert = cert - - if certificate_store.verify(@last_seen_cert) - begin - certificate_store.add_cert(@last_seen_cert) - rescue OpenSSL::X509::StoreError => e - raise e unless e.message == 'cert already in hash table' - end - true - else - raise OpenSSL::SSL::SSLError.new(%(unable to verify the server certificate for "#{host}")) - end - end - - def ssl_handshake_completed - return true unless verify_peer? - - unless OpenSSL::SSL.verify_certificate_identity(@last_seen_cert, host) - raise OpenSSL::SSL::SSLError.new(%(host "#{host}" does not match the server certificate)) - else - true - end - end - - def verify_peer? - parent.connopts.tls[:verify_peer] - end - - def host - parent.connopts.host - end - - def certificate_store - @certificate_store ||= begin - store = OpenSSL::X509::Store.new - store.set_default_paths - ca_file = parent.connopts.tls[:cert_chain_file] - store.add_file(ca_file) if ca_file - store - end - end -end - -EventMachine::HttpStubConnection.send(:include, EmHttpSslPatch) diff --git a/aviator/lib/puppet/feature/faraday/adapter/em_synchrony.rb b/aviator/lib/puppet/feature/faraday/adapter/em_synchrony.rb deleted file mode 100644 index 305e702f8..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/em_synchrony.rb +++ /dev/null @@ -1,92 +0,0 @@ -require 'uri' - -module Faraday - class Adapter - class EMSynchrony < Faraday::Adapter - include EMHttp::Options - - dependency do - require 'em-synchrony/em-http' - require 'em-synchrony/em-multi' - require 'fiber' - end - - self.supports_parallel = true - - def self.setup_parallel_manager(options = {}) - ParallelManager.new - end - - def call(env) - super - request = EventMachine::HttpRequest.new(Utils::URI(env[:url].to_s), connection_config(env)) - - http_method = env[:method].to_s.downcase.to_sym - - # Queue requests for parallel execution. - if env[:parallel_manager] - env[:parallel_manager].add(request, http_method, request_config(env)) do |resp| - save_response(env, resp.response_header.status, resp.response) do |resp_headers| - resp.response_header.each do |name, value| - resp_headers[name.to_sym] = value - end - end - - # Finalize the response object with values from `env`. - env[:response].finish(env) - end - - # Execute single request. - else - client = nil - block = lambda { request.send(http_method, request_config(env)) } - - if !EM.reactor_running? - EM.run do - Fiber.new { - client = block.call - EM.stop - }.resume - end - else - client = block.call - end - - raise client.error if client.error - - save_response(env, client.response_header.status, client.response) do |resp_headers| - client.response_header.each do |name, value| - resp_headers[name.to_sym] = value - end - end - end - - @app.call env - rescue Errno::ECONNREFUSED - raise Error::ConnectionFailed, $! - rescue EventMachine::Connectify::CONNECTError => err - if err.message.include?("Proxy Authentication Required") - raise Error::ConnectionFailed, %{407 "Proxy Authentication Required "} - else - raise Error::ConnectionFailed, err - end - rescue => err - if defined?(OpenSSL) && OpenSSL::SSL::SSLError === err - raise Faraday::SSLError, err - else - raise - end - end - end - end -end - -require 'faraday/adapter/em_synchrony/parallel_manager' - -begin - require 'openssl' -rescue LoadError - warn "Warning: no such file to load -- openssl. Make sure it is installed if you want HTTPS support" -else - require 'faraday/adapter/em_http_ssl_patch' -end if Faraday::Adapter::EMSynchrony.loaded? diff --git a/aviator/lib/puppet/feature/faraday/adapter/em_synchrony/parallel_manager.rb b/aviator/lib/puppet/feature/faraday/adapter/em_synchrony/parallel_manager.rb deleted file mode 100644 index 12a1bafa4..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/em_synchrony/parallel_manager.rb +++ /dev/null @@ -1,66 +0,0 @@ -module Faraday - class Adapter - class EMSynchrony < Faraday::Adapter - class ParallelManager - - # Add requests to queue. The `request` argument should be a - # `EM::HttpRequest` object. - def add(request, method, *args, &block) - queue << { - :request => request, - :method => method, - :args => args, - :block => block - } - end - - # Run all requests on queue with `EM::Synchrony::Multi`, wrapping - # it in a reactor and fiber if needed. - def run - result = nil - if !EM.reactor_running? - EM.run { - Fiber.new do - result = perform - EM.stop - end.resume - } - else - result = perform - end - result - end - - - private - - # The request queue. - def queue - @queue ||= [] - end - - # Main `EM::Synchrony::Multi` performer. - def perform - multi = ::EM::Synchrony::Multi.new - - queue.each do |item| - method = "a#{item[:method]}".to_sym - - req = item[:request].send(method, *item[:args]) - req.callback(&item[:block]) - - req_name = "req_#{multi.requests.size}".to_sym - multi.add(req_name, req) - end - - # Clear the queue, so parallel manager objects can be reused. - @queue = [] - - # Block fiber until all requests have returned. - multi.perform - end - - end # ParallelManager - end # EMSynchrony - end # Adapter -end # Faraday diff --git a/aviator/lib/puppet/feature/faraday/adapter/excon.rb b/aviator/lib/puppet/feature/faraday/adapter/excon.rb deleted file mode 100644 index db0c7c352..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/excon.rb +++ /dev/null @@ -1,80 +0,0 @@ -module Faraday - class Adapter - class Excon < Faraday::Adapter - dependency 'excon' - - def initialize(app, connection_options = {}) - @connection_options = connection_options - super(app) - end - - def call(env) - super - - opts = {} - if env[:url].scheme == 'https' && ssl = env[:ssl] - opts[:ssl_verify_peer] = !!ssl.fetch(:verify, true) - opts[:ssl_ca_path] = ssl[:ca_path] if ssl[:ca_path] - opts[:ssl_ca_file] = ssl[:ca_file] if ssl[:ca_file] - opts[:client_cert] = ssl[:client_cert] if ssl[:client_cert] - opts[:client_key] = ssl[:client_key] if ssl[:client_key] - opts[:certificate] = ssl[:certificate] if ssl[:certificate] - opts[:private_key] = ssl[:private_key] if ssl[:private_key] - - # https://github.com/geemus/excon/issues/106 - # https://github.com/jruby/jruby-ossl/issues/19 - opts[:nonblock] = false - end - - if ( req = env[:request] ) - if req[:timeout] - opts[:read_timeout] = req[:timeout] - opts[:connect_timeout] = req[:timeout] - opts[:write_timeout] = req[:timeout] - end - - if req[:open_timeout] - opts[:connect_timeout] = req[:open_timeout] - opts[:write_timeout] = req[:open_timeout] - end - - if req[:proxy] - opts[:proxy] = { - :host => req[:proxy][:uri].host, - :port => req[:proxy][:uri].port, - :scheme => req[:proxy][:uri].scheme, - :user => req[:proxy][:user], - :password => req[:proxy][:password] - } - end - end - - conn = ::Excon.new(env[:url].to_s, opts.merge(@connection_options)) - - resp = conn.request \ - :method => env[:method].to_s.upcase, - :headers => env[:request_headers], - :body => read_body(env) - - save_response(env, resp.status.to_i, resp.body, resp.headers) - - @app.call env - rescue ::Excon::Errors::SocketError => err - if err.message =~ /\btimeout\b/ - raise Error::TimeoutError, err - elsif err.message =~ /\bcertificate\b/ - raise Faraday::SSLError, err - else - raise Error::ConnectionFailed, err - end - rescue ::Excon::Errors::Timeout => err - raise Error::TimeoutError, err - end - - # TODO: support streaming requests - def read_body(env) - env[:body].respond_to?(:read) ? env[:body].read : env[:body] - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/adapter/httpclient.rb b/aviator/lib/puppet/feature/faraday/adapter/httpclient.rb deleted file mode 100644 index 06c663f03..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/httpclient.rb +++ /dev/null @@ -1,106 +0,0 @@ -module Faraday - class Adapter - class HTTPClient < Faraday::Adapter - dependency 'httpclient' - - def client - @client ||= ::HTTPClient.new - end - - def call(env) - super - - if req = env[:request] - if proxy = req[:proxy] - configure_proxy proxy - end - - if bind = req[:bind] - configure_socket bind - end - - configure_timeouts req - end - - if env[:url].scheme == 'https' && ssl = env[:ssl] - configure_ssl ssl - end - - # TODO Don't stream yet. - # https://github.com/nahi/httpclient/pull/90 - env[:body] = env[:body].read if env[:body].respond_to? :read - - resp = client.request env[:method], env[:url], - :body => env[:body], - :header => env[:request_headers] - - save_response env, resp.status, resp.body, resp.headers - - @app.call env - rescue ::HTTPClient::TimeoutError - raise Faraday::Error::TimeoutError, $! - rescue ::HTTPClient::BadResponseError => err - if err.message.include?('status 407') - raise Faraday::Error::ConnectionFailed, %{407 "Proxy Authentication Required "} - else - raise Faraday::Error::ClientError, $! - end - rescue Errno::ECONNREFUSED, EOFError - raise Faraday::Error::ConnectionFailed, $! - rescue => err - if defined?(OpenSSL) && OpenSSL::SSL::SSLError === err - raise Faraday::SSLError, err - else - raise - end - end - - def configure_socket(bind) - client.socket_local.host = bind[:host] - client.socket_local.port = bind[:port] - end - - def configure_proxy(proxy) - client.proxy = proxy[:uri] - if proxy[:user] && proxy[:password] - client.set_proxy_auth proxy[:user], proxy[:password] - end - end - - def configure_ssl(ssl) - ssl_config = client.ssl_config - - ssl_config.add_trust_ca ssl[:ca_file] if ssl[:ca_file] - ssl_config.add_trust_ca ssl[:ca_path] if ssl[:ca_path] - ssl_config.cert_store = ssl[:cert_store] if ssl[:cert_store] - ssl_config.client_cert = ssl[:client_cert] if ssl[:client_cert] - ssl_config.client_key = ssl[:client_key] if ssl[:client_key] - ssl_config.verify_depth = ssl[:verify_depth] if ssl[:verify_depth] - ssl_config.verify_mode = ssl_verify_mode(ssl) - end - - def configure_timeouts(req) - if req[:timeout] - client.connect_timeout = req[:timeout] - client.receive_timeout = req[:timeout] - client.send_timeout = req[:timeout] - end - - if req[:open_timeout] - client.connect_timeout = req[:open_timeout] - client.send_timeout = req[:open_timeout] - end - end - - def ssl_verify_mode(ssl) - ssl[:verify_mode] || begin - if ssl.fetch(:verify, true) - OpenSSL::SSL::VERIFY_PEER | OpenSSL::SSL::VERIFY_FAIL_IF_NO_PEER_CERT - else - OpenSSL::SSL::VERIFY_NONE - end - end - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/adapter/net_http.rb b/aviator/lib/puppet/feature/faraday/adapter/net_http.rb deleted file mode 100644 index 449388a7f..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/net_http.rb +++ /dev/null @@ -1,124 +0,0 @@ -begin - require 'net/https' -rescue LoadError - warn "Warning: no such file to load -- net/https. Make sure openssl is installed if you want ssl support" - require 'net/http' -end -require 'zlib' - -module Faraday - class Adapter - class NetHttp < Faraday::Adapter - NET_HTTP_EXCEPTIONS = [ - EOFError, - Errno::ECONNABORTED, - Errno::ECONNREFUSED, - Errno::ECONNRESET, - Errno::EHOSTUNREACH, - Errno::EINVAL, - Errno::ENETUNREACH, - Net::HTTPBadResponse, - Net::HTTPHeaderSyntaxError, - Net::ProtocolError, - SocketError, - Zlib::GzipFile::Error, - ] - - NET_HTTP_EXCEPTIONS << OpenSSL::SSL::SSLError if defined?(OpenSSL) - - def call(env) - super - http = net_http_connection(env) - configure_ssl(http, env[:ssl]) if env[:url].scheme == 'https' and env[:ssl] - - req = env[:request] - http.read_timeout = http.open_timeout = req[:timeout] if req[:timeout] - http.open_timeout = req[:open_timeout] if req[:open_timeout] - - begin - http_response = perform_request(http, env) - rescue *NET_HTTP_EXCEPTIONS => err - if defined?(OpenSSL) && OpenSSL::SSL::SSLError === err - raise Faraday::SSLError, err - else - raise Error::ConnectionFailed, err - end - end - - save_response(env, http_response.code.to_i, http_response.body || '') do |response_headers| - http_response.each_header do |key, value| - response_headers[key] = value - end - end - - @app.call env - rescue Timeout::Error => err - raise Faraday::Error::TimeoutError, err - end - - def create_request(env) - request = Net::HTTPGenericRequest.new \ - env[:method].to_s.upcase, # request method - !!env[:body], # is there request body - :head != env[:method], # is there response body - env[:url].request_uri, # request uri path - env[:request_headers] # request headers - - if env[:body].respond_to?(:read) - request.body_stream = env[:body] - else - request.body = env[:body] - end - request - end - - def perform_request(http, env) - if :get == env[:method] and !env[:body] - # prefer `get` to `request` because the former handles gzip (ruby 1.9) - http.get env[:url].request_uri, env[:request_headers] - else - http.request create_request(env) - end - end - - def net_http_connection(env) - if proxy = env[:request][:proxy] - Net::HTTP::Proxy(proxy[:uri].host, proxy[:uri].port, proxy[:user], proxy[:password]) - else - Net::HTTP - end.new(env[:url].host, env[:url].port) - end - - def configure_ssl(http, ssl) - http.use_ssl = true - http.verify_mode = ssl_verify_mode(ssl) - http.cert_store = ssl_cert_store(ssl) - - http.cert = ssl[:client_cert] if ssl[:client_cert] - http.key = ssl[:client_key] if ssl[:client_key] - http.ca_file = ssl[:ca_file] if ssl[:ca_file] - http.ca_path = ssl[:ca_path] if ssl[:ca_path] - http.verify_depth = ssl[:verify_depth] if ssl[:verify_depth] - http.ssl_version = ssl[:version] if ssl[:version] - end - - def ssl_cert_store(ssl) - return ssl[:cert_store] if ssl[:cert_store] - # Use the default cert store by default, i.e. system ca certs - cert_store = OpenSSL::X509::Store.new - cert_store.set_default_paths - cert_store - end - - def ssl_verify_mode(ssl) - ssl[:verify_mode] || begin - if ssl.fetch(:verify, true) - OpenSSL::SSL::VERIFY_PEER - else - OpenSSL::SSL::VERIFY_NONE - end - end - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/adapter/net_http_persistent.rb b/aviator/lib/puppet/feature/faraday/adapter/net_http_persistent.rb deleted file mode 100644 index e0cc9587e..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/net_http_persistent.rb +++ /dev/null @@ -1,47 +0,0 @@ -# Rely on autoloading instead of explicit require; helps avoid the "already -# initialized constant" warning on Ruby 1.8.7 when NetHttp is refereced below. -# require 'faraday/adapter/net_http' - -module Faraday - class Adapter - # Experimental adapter for net-http-persistent - class NetHttpPersistent < NetHttp - dependency 'net/http/persistent' - - def net_http_connection(env) - if proxy = env[:request][:proxy] - proxy_uri = ::URI::HTTP === proxy[:uri] ? proxy[:uri].dup : ::URI.parse(proxy[:uri].to_s) - proxy_uri.user = proxy_uri.password = nil - # awful patch for net-http-persistent 2.8 not unescaping user/password - (class << proxy_uri; self; end).class_eval do - define_method(:user) { proxy[:user] } - define_method(:password) { proxy[:password] } - end if proxy[:user] - end - Net::HTTP::Persistent.new 'Faraday', proxy_uri - end - - def perform_request(http, env) - http.request env[:url], create_request(env) - rescue Net::HTTP::Persistent::Error => error - if error.message.include? 'Timeout' - raise Faraday::Error::TimeoutError, error - elsif error.message.include? 'connection refused' - raise Faraday::Error::ConnectionFailed, error - else - raise - end - end - - def configure_ssl(http, ssl) - http.verify_mode = ssl_verify_mode(ssl) - http.cert_store = ssl_cert_store(ssl) - - http.certificate = ssl[:client_cert] if ssl[:client_cert] - http.private_key = ssl[:client_key] if ssl[:client_key] - http.ca_file = ssl[:ca_file] if ssl[:ca_file] - http.ssl_version = ssl[:version] if ssl[:version] - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/adapter/patron.rb b/aviator/lib/puppet/feature/faraday/adapter/patron.rb deleted file mode 100644 index cf2d37fbe..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/patron.rb +++ /dev/null @@ -1,72 +0,0 @@ -module Faraday - class Adapter - class Patron < Faraday::Adapter - dependency 'patron' - - def initialize(app, &block) - super(app) - @block = block - end - - def call(env) - super - - # TODO: support streaming requests - env[:body] = env[:body].read if env[:body].respond_to? :read - - session = @session ||= create_session - - if req = env[:request] - session.timeout = session.connect_timeout = req[:timeout] if req[:timeout] - session.connect_timeout = req[:open_timeout] if req[:open_timeout] - - if proxy = req[:proxy] - proxy_uri = proxy[:uri].dup - proxy_uri.user = proxy[:user] && Utils.escape(proxy[:user]).gsub('+', '%20') - proxy_uri.password = proxy[:password] && Utils.escape(proxy[:password]).gsub('+', '%20') - session.proxy = proxy_uri.to_s - end - end - - response = begin - data = env[:body] ? env[:body].to_s : nil - session.request(env[:method], env[:url].to_s, env[:request_headers], :data => data) - rescue Errno::ECONNREFUSED, ::Patron::ConnectionFailed - raise Error::ConnectionFailed, $! - end - - save_response(env, response.status, response.body, response.headers) - - @app.call env - rescue ::Patron::TimeoutError => err - if err.message == "Connection time-out" - raise Faraday::Error::ConnectionFailed, err - else - raise Faraday::Error::TimeoutError, err - end - rescue ::Patron::Error => err - if err.message.include?("code 407") - raise Error::ConnectionFailed, %{407 "Proxy Authentication Required "} - else - raise Error::ConnectionFailed, err - end - end - - if loaded? && defined?(::Patron::Request::VALID_ACTIONS) - # HAX: helps but doesn't work completely - # https://github.com/toland/patron/issues/34 - ::Patron::Request::VALID_ACTIONS.tap do |actions| - actions << :patch unless actions.include? :patch - actions << :options unless actions.include? :options - end - end - - def create_session - session = ::Patron::Session.new - session.insecure = true - @block.call(session) if @block - session - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/adapter/rack.rb b/aviator/lib/puppet/feature/faraday/adapter/rack.rb deleted file mode 100644 index 0d2146487..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/rack.rb +++ /dev/null @@ -1,58 +0,0 @@ -module Faraday - class Adapter - # Sends requests to a Rack app. - # - # Examples - # - # class MyRackApp - # def call(env) - # [200, {'Content-Type' => 'text/html'}, ["hello world"]] - # end - # end - # - # Faraday.new do |conn| - # conn.adapter :rack, MyRackApp.new - # end - class Rack < Faraday::Adapter - dependency 'rack/test' - - # not prefixed with "HTTP_" - SPECIAL_HEADERS = %w[ CONTENT_LENGTH CONTENT_TYPE ] - - def initialize(faraday_app, rack_app) - super(faraday_app) - mock_session = ::Rack::MockSession.new(rack_app) - @session = ::Rack::Test::Session.new(mock_session) - end - - def call(env) - super - rack_env = { - :method => env[:method], - :input => env[:body].respond_to?(:read) ? env[:body].read : env[:body], - 'rack.url_scheme' => env[:url].scheme - } - - env[:request_headers].each do |name, value| - name = name.upcase.tr('-', '_') - name = "HTTP_#{name}" unless SPECIAL_HEADERS.include? name - rack_env[name] = value - end if env[:request_headers] - - timeout = env[:request][:timeout] || env[:request][:open_timeout] - response = if timeout - Timer.timeout(timeout, Faraday::Error::TimeoutError) { execute_request(env, rack_env) } - else - execute_request(env, rack_env) - end - - save_response(env, response.status, response.body, response.headers) - @app.call env - end - - def execute_request(env, rack_env) - @session.request(env[:url].to_s, rack_env) - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/adapter/test.rb b/aviator/lib/puppet/feature/faraday/adapter/test.rb deleted file mode 100644 index 9a345758e..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/test.rb +++ /dev/null @@ -1,162 +0,0 @@ -module Faraday - class Adapter - # test = Faraday::Connection.new do - # use Faraday::Adapter::Test do |stub| - # stub.get '/nigiri/sake.json' do - # [200, {}, 'hi world'] - # end - # end - # end - # - # resp = test.get '/nigiri/sake.json' - # resp.body # => 'hi world' - # - class Test < Faraday::Adapter - attr_accessor :stubs - - class Stubs - class NotFound < StandardError - end - - def initialize - # {:get => [Stub, Stub]} - @stack, @consumed = {}, {} - yield(self) if block_given? - end - - def empty? - @stack.empty? - end - - def match(request_method, path, headers, body) - return false if !@stack.key?(request_method) - stack = @stack[request_method] - consumed = (@consumed[request_method] ||= []) - - if stub = matches?(stack, path, headers, body) - consumed << stack.delete(stub) - stub - else - matches?(consumed, path, headers, body) - end - end - - def get(path, headers = {}, &block) - new_stub(:get, path, headers, &block) - end - - def head(path, headers = {}, &block) - new_stub(:head, path, headers, &block) - end - - def post(path, body=nil, headers = {}, &block) - new_stub(:post, path, headers, body, &block) - end - - def put(path, body=nil, headers = {}, &block) - new_stub(:put, path, headers, body, &block) - end - - def patch(path, body=nil, headers = {}, &block) - new_stub(:patch, path, headers, body, &block) - end - - def delete(path, headers = {}, &block) - new_stub(:delete, path, headers, &block) - end - - def options(path, headers = {}, &block) - new_stub(:options, path, headers, &block) - end - - # Raises an error if any of the stubbed calls have not been made. - def verify_stubbed_calls - failed_stubs = [] - @stack.each do |method, stubs| - unless stubs.size == 0 - failed_stubs.concat(stubs.map {|stub| - "Expected #{method} #{stub}." - }) - end - end - raise failed_stubs.join(" ") unless failed_stubs.size == 0 - end - - protected - - def new_stub(request_method, path, headers = {}, body=nil, &block) - normalized_path = Faraday::Utils.normalize_path(path) - (@stack[request_method] ||= []) << Stub.new(normalized_path, headers, body, block) - end - - def matches?(stack, path, headers, body) - stack.detect { |stub| stub.matches?(path, headers, body) } - end - end - - class Stub < Struct.new(:path, :params, :headers, :body, :block) - def initialize(full, headers, body, block) - path, query = full.split('?') - params = query ? - Faraday::Utils.parse_nested_query(query) : - {} - super(path, params, headers, body, block) - end - - def matches?(request_uri, request_headers, request_body) - request_path, request_query = request_uri.split('?') - request_params = request_query ? - Faraday::Utils.parse_nested_query(request_query) : - {} - request_path == path && - params_match?(request_params) && - (body.to_s.size.zero? || request_body == body) && - headers_match?(request_headers) - end - - def params_match?(request_params) - params.keys.all? do |key| - request_params[key] == params[key] - end - end - - def headers_match?(request_headers) - headers.keys.all? do |key| - request_headers[key] == headers[key] - end - end - - def to_s - "#{path} #{body}" - end - end - - def initialize(app, stubs=nil, &block) - super(app) - @stubs = stubs || Stubs.new - configure(&block) if block - end - - def configure - yield(stubs) - end - - def call(env) - super - normalized_path = Faraday::Utils.normalize_path(env[:url]) - params_encoder = env.request.params_encoder || Faraday::Utils.default_params_encoder - - if stub = stubs.match(env[:method], normalized_path, env.request_headers, env[:body]) - env[:params] = (query = env[:url].query) ? - params_encoder.decode(query) : - {} - status, headers, body = stub.block.call(env) - save_response(env, status, body, headers) - else - raise Stubs::NotFound, "no stubbed request for #{env[:method]} #{normalized_path} #{env[:body]}" - end - @app.call(env) - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/adapter/typhoeus.rb b/aviator/lib/puppet/feature/faraday/adapter/typhoeus.rb deleted file mode 100644 index 69b6a5139..000000000 --- a/aviator/lib/puppet/feature/faraday/adapter/typhoeus.rb +++ /dev/null @@ -1,123 +0,0 @@ -module Faraday - class Adapter - class Typhoeus < Faraday::Adapter - self.supports_parallel = true - - def self.setup_parallel_manager(options = {}) - options.empty? ? ::Typhoeus::Hydra.hydra : ::Typhoeus::Hydra.new(options) - end - - dependency 'typhoeus' - - def call(env) - super - perform_request env - @app.call env - end - - def perform_request(env) - read_body env - - hydra = env[:parallel_manager] || self.class.setup_parallel_manager - hydra.queue request(env) - hydra.run unless parallel?(env) - rescue Errno::ECONNREFUSED - raise Error::ConnectionFailed, $! - end - - # TODO: support streaming requests - def read_body(env) - env[:body] = env[:body].read if env[:body].respond_to? :read - end - - def request(env) - method = env[:method] - # For some reason, prevents Typhoeus from using "100-continue". - # We want this because Webrick 1.3.1 can't seem to handle it w/ PUT. - method = method.to_s.upcase if method == :put - - req = ::Typhoeus::Request.new env[:url].to_s, - :method => method, - :body => env[:body], - :headers => env[:request_headers], - :disable_ssl_peer_verification => (env[:ssl] && env[:ssl].disable?) - - configure_ssl req, env - configure_proxy req, env - configure_timeout req, env - configure_socket req, env - - req.on_complete do |resp| - if resp.timed_out? - if parallel?(env) - # TODO: error callback in async mode - else - raise Faraday::Error::TimeoutError, "request timed out" - end - end - - case resp.curl_return_code - when 0 - # everything OK - when 7 - raise Error::ConnectionFailed, resp.curl_error_message - when 60 - raise Faraday::SSLError, resp.curl_error_message - else - raise Error::ClientError, resp.curl_error_message - end - - save_response(env, resp.code, resp.body) do |response_headers| - response_headers.parse resp.headers - end - # in async mode, :response is initialized at this point - env[:response].finish(env) if parallel?(env) - end - - req - end - - def configure_ssl(req, env) - ssl = env[:ssl] - - req.ssl_version = ssl[:version] if ssl[:version] - req.ssl_cert = ssl[:client_cert] if ssl[:client_cert] - req.ssl_key = ssl[:client_key] if ssl[:client_key] - req.ssl_cacert = ssl[:ca_file] if ssl[:ca_file] - req.ssl_capath = ssl[:ca_path] if ssl[:ca_path] - end - - def configure_proxy(req, env) - proxy = request_options(env)[:proxy] - return unless proxy - - req.proxy = "#{proxy[:uri].host}:#{proxy[:uri].port}" - - if proxy[:user] && proxy[:password] - req.proxy_username = proxy[:user] - req.proxy_password = proxy[:password] - end - end - - def configure_timeout(req, env) - env_req = request_options(env) - req.timeout = req.connect_timeout = (env_req[:timeout] * 1000) if env_req[:timeout] - req.connect_timeout = (env_req[:open_timeout] * 1000) if env_req[:open_timeout] - end - - def configure_socket(req, env) - if bind = request_options(env)[:bind] - req.interface = bind[:host] - end - end - - def request_options(env) - env[:request] - end - - def parallel?(env) - !!env[:parallel_manager] - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/autoload.rb b/aviator/lib/puppet/feature/faraday/autoload.rb deleted file mode 100644 index ec413ff84..000000000 --- a/aviator/lib/puppet/feature/faraday/autoload.rb +++ /dev/null @@ -1,85 +0,0 @@ -module Faraday - # Internal: Adds the ability for other modules to manage autoloadable - # constants. - module AutoloadHelper - # Internal: Registers the constants to be auto loaded. - # - # prefix - The String require prefix. If the path is inside Faraday, then - # it will be prefixed with the root path of this loaded Faraday - # version. - # options - Hash of Symbol => String library names. - # - # Examples. - # - # Faraday.autoload_all 'faraday/foo', - # :Bar => 'bar' - # - # # requires faraday/foo/bar to load Faraday::Bar. - # Faraday::Bar - # - # - # Returns nothing. - def autoload_all(prefix, options) - if prefix =~ /^faraday(\/|$)/i - prefix = File.join(Faraday.root_path, prefix) - end - options.each do |const_name, path| - autoload const_name, File.join(prefix, path) - end - end - - # Internal: Loads each autoloaded constant. If thread safety is a concern, - # wrap this in a Mutex. - # - # Returns nothing. - def load_autoloaded_constants - constants.each do |const| - const_get(const) if autoload?(const) - end - end - - # Internal: Filters the module's contents with those that have been already - # autoloaded. - # - # Returns an Array of Class/Module objects. - def all_loaded_constants - constants.map { |c| const_get(c) }. - select { |a| a.respond_to?(:loaded?) && a.loaded? } - end - end - - class Adapter - extend AutoloadHelper - autoload_all 'faraday/adapter', - :NetHttp => 'net_http', - :NetHttpPersistent => 'net_http_persistent', - :Typhoeus => 'typhoeus', - :EMSynchrony => 'em_synchrony', - :EMHttp => 'em_http', - :Patron => 'patron', - :Excon => 'excon', - :Test => 'test', - :Rack => 'rack', - :HTTPClient => 'httpclient' - end - - class Request - extend AutoloadHelper - autoload_all 'faraday/request', - :UrlEncoded => 'url_encoded', - :Multipart => 'multipart', - :Retry => 'retry', - :Timeout => 'timeout', - :Authorization => 'authorization', - :BasicAuthentication => 'basic_authentication', - :TokenAuthentication => 'token_authentication', - :Instrumentation => 'instrumentation' - end - - class Response - extend AutoloadHelper - autoload_all 'faraday/response', - :RaiseError => 'raise_error', - :Logger => 'logger' - end -end diff --git a/aviator/lib/puppet/feature/faraday/connection.rb b/aviator/lib/puppet/feature/faraday/connection.rb deleted file mode 100644 index 1e408e2c2..000000000 --- a/aviator/lib/puppet/feature/faraday/connection.rb +++ /dev/null @@ -1,432 +0,0 @@ -module Faraday - # Public: Connection objects manage the default properties and the middleware - # stack for fulfilling an HTTP request. - # - # Examples - # - # conn = Faraday::Connection.new 'http://sushi.com' - # - # # GET http://sushi.com/nigiri - # conn.get 'nigiri' - # # => # - # - class Connection - # A Set of allowed HTTP verbs. - METHODS = Set.new [:get, :post, :put, :delete, :head, :patch, :options] - - # Public: Returns a Hash of URI query unencoded key/value pairs. - attr_reader :params - - # Public: Returns a Hash of unencoded HTTP header key/value pairs. - attr_reader :headers - - # Public: Returns a URI with the prefix used for all requests from this - # Connection. This includes a default host name, scheme, port, and path. - attr_reader :url_prefix - - # Public: Returns the Faraday::Builder for this Connection. - attr_reader :builder - - # Public: Returns a Hash of the request options. - attr_reader :options - - # Public: Returns a Hash of the SSL options. - attr_reader :ssl - - # Public: Returns the parallel manager for this Connection. - attr_reader :parallel_manager - - # Public: Sets the default parallel manager for this connection. - attr_writer :default_parallel_manager - - # Public: Initializes a new Faraday::Connection. - # - # url - URI or String base URL to use as a prefix for all - # requests (optional). - # options - Hash or Faraday::ConnectionOptions. - # :url - URI or String base URL (default: "http:/"). - # :params - Hash of URI query unencoded key/value pairs. - # :headers - Hash of unencoded HTTP header key/value pairs. - # :request - Hash of request options. - # :ssl - Hash of SSL options. - # :proxy - URI, String or Hash of HTTP proxy options - # (default: "http_proxy" environment variable). - # :uri - URI or String - # :user - String (optional) - # :password - String (optional) - def initialize(url = nil, options = nil) - if url.is_a?(Hash) - options = ConnectionOptions.from(url) - url = options.url - else - options = ConnectionOptions.from(options) - end - - @parallel_manager = nil - @headers = Utils::Headers.new - @params = Utils::ParamsHash.new - @options = options.request - @ssl = options.ssl - @default_parallel_manager = options.parallel_manager - - @builder = options.builder || begin - # pass an empty block to Builder so it doesn't assume default middleware - options.new_builder(block_given? ? Proc.new { |b| } : nil) - end - - self.url_prefix = url || 'http:/' - - @params.update(options.params) if options.params - @headers.update(options.headers) if options.headers - - @proxy = nil - proxy(options.fetch(:proxy) { - uri = ENV['http_proxy'] - if uri && !uri.empty? - uri = 'http://' + uri if uri !~ /^http/i - uri - end - }) - - yield(self) if block_given? - - @headers[:user_agent] ||= "Faraday v#{VERSION}" - end - - # Public: Sets the Hash of URI query unencoded key/value pairs. - def params=(hash) - @params.replace hash - end - - # Public: Sets the Hash of unencoded HTTP header key/value pairs. - def headers=(hash) - @headers.replace hash - end - - extend Forwardable - - def_delegators :builder, :build, :use, :request, :response, :adapter, :app - - # Public: Makes an HTTP request without a body. - # - # url - The optional String base URL to use as a prefix for all - # requests. Can also be the options Hash. - # params - Hash of URI query unencoded key/value pairs. - # headers - Hash of unencoded HTTP header key/value pairs. - # - # Examples - # - # conn.get '/items', {:page => 1}, :accept => 'application/json' - # conn.head '/items/1' - # - # # ElasticSearch example sending a body with GET. - # conn.get '/twitter/tweet/_search' do |req| - # req.headers[:content_type] = 'application/json' - # req.params[:routing] = 'kimchy' - # req.body = JSON.generate(:query => {...}) - # end - # - # Yields a Faraday::Response for further request customizations. - # Returns a Faraday::Response. - # - # Signature - # - # (url = nil, params = nil, headers = nil) - # - # verb - An HTTP verb: get, head, or delete. - %w[get head delete].each do |method| - class_eval <<-RUBY, __FILE__, __LINE__ + 1 - def #{method}(url = nil, params = nil, headers = nil) - run_request(:#{method}, url, nil, headers) { |request| - request.params.update(params) if params - yield(request) if block_given? - } - end - RUBY - end - - # Public: Makes an HTTP request with a body. - # - # url - The optional String base URL to use as a prefix for all - # requests. Can also be the options Hash. - # body - The String body for the request. - # headers - Hash of unencoded HTTP header key/value pairs. - # - # Examples - # - # conn.post '/items', data, :content_type => 'application/json' - # - # # Simple ElasticSearch indexing sample. - # conn.post '/twitter/tweet' do |req| - # req.headers[:content_type] = 'application/json' - # req.params[:routing] = 'kimchy' - # req.body = JSON.generate(:user => 'kimchy', ...) - # end - # - # Yields a Faraday::Response for further request customizations. - # Returns a Faraday::Response. - # - # Signature - # - # (url = nil, body = nil, headers = nil) - # - # verb - An HTTP verb: post, put, or patch. - %w[post put patch].each do |method| - class_eval <<-RUBY, __FILE__, __LINE__ + 1 - def #{method}(url = nil, body = nil, headers = nil, &block) - run_request(:#{method}, url, body, headers, &block) - end - RUBY - end - - # Public: Sets up the Authorization header with these credentials, encoded - # with base64. - # - # login - The authentication login. - # pass - The authentication password. - # - # Examples - # - # conn.basic_auth 'Aladdin', 'open sesame' - # conn.headers['Authorization'] - # # => "Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==" - # - # Returns nothing. - def basic_auth(login, pass) - set_authorization_header(:basic_auth, login, pass) - end - - # Public: Sets up the Authorization header with the given token. - # - # token - The String token. - # options - Optional Hash of extra token options. - # - # Examples - # - # conn.token_auth 'abcdef', :foo => 'bar' - # conn.headers['Authorization'] - # # => "Token token=\"abcdef\", - # foo=\"bar\"" - # - # Returns nothing. - def token_auth(token, options = nil) - set_authorization_header(:token_auth, token, options) - end - - # Public: Sets up a custom Authorization header. - # - # type - The String authorization type. - # token - The String or Hash token. A String value is taken literally, and - # a Hash is encoded into comma separated key/value pairs. - # - # Examples - # - # conn.authorization :Bearer, 'mF_9.B5f-4.1JqM' - # conn.headers['Authorization'] - # # => "Bearer mF_9.B5f-4.1JqM" - # - # conn.authorization :Token, :token => 'abcdef', :foo => 'bar' - # conn.headers['Authorization'] - # # => "Token token=\"abcdef\", - # foo=\"bar\"" - # - # Returns nothing. - def authorization(type, token) - set_authorization_header(:authorization, type, token) - end - - # Internal: Traverse the middleware stack in search of a - # parallel-capable adapter. - # - # Yields in case of not found. - # - # Returns a parallel manager or nil if not found. - def default_parallel_manager - @default_parallel_manager ||= begin - handler = @builder.handlers.detect do |h| - h.klass.respond_to?(:supports_parallel?) and h.klass.supports_parallel? - end - - if handler - handler.klass.setup_parallel_manager - elsif block_given? - yield - end - end - end - - # Public: Determine if this Faraday::Connection can make parallel requests. - # - # Returns true or false. - def in_parallel? - !!@parallel_manager - end - - # Public: Sets up the parallel manager to make a set of requests. - # - # manager - The parallel manager that this Connection's Adapter uses. - # - # Yields a block to execute multiple requests. - # Returns nothing. - def in_parallel(manager = nil) - @parallel_manager = manager || default_parallel_manager { - warn "Warning: `in_parallel` called but no parallel-capable adapter on Faraday stack" - warn caller[2,10].join("\n") - nil - } - yield - @parallel_manager && @parallel_manager.run - ensure - @parallel_manager = nil - end - - # Public: Gets or Sets the Hash proxy options. - def proxy(arg = nil) - return @proxy if arg.nil? - @proxy = ProxyOptions.from(arg) - end - - def_delegators :url_prefix, :scheme, :scheme=, :host, :host=, :port, :port= - def_delegator :url_prefix, :path, :path_prefix - - # Public: Parses the giving url with URI and stores the individual - # components in this connection. These components serve as defaults for - # requests made by this connection. - # - # url - A String or URI. - # - # Examples - # - # conn = Faraday::Connection.new { ... } - # conn.url_prefix = "https://sushi.com/api" - # conn.scheme # => https - # conn.path_prefix # => "/api" - # - # conn.get("nigiri?page=2") # accesses https://sushi.com/api/nigiri - # - # Returns the parsed URI from teh given input.. - def url_prefix=(url, encoder = nil) - uri = @url_prefix = Utils.URI(url) - self.path_prefix = uri.path - - params.merge_query(uri.query, encoder) - uri.query = nil - - with_uri_credentials(uri) do |user, password| - basic_auth user, password - uri.user = uri.password = nil - end - - uri - end - - # Public: Sets the path prefix and ensures that it always has a leading - # slash. - # - # value - A String. - # - # Returns the new String path prefix. - def path_prefix=(value) - url_prefix.path = if value - value = '/' + value unless value[0,1] == '/' - value - end - end - - # Public: Takes a relative url for a request and combines it with the defaults - # set on the connection instance. - # - # conn = Faraday::Connection.new { ... } - # conn.url_prefix = "https://sushi.com/api?token=abc" - # conn.scheme # => https - # conn.path_prefix # => "/api" - # - # conn.build_url("nigiri?page=2") # => https://sushi.com/api/nigiri?token=abc&page=2 - # conn.build_url("nigiri", :page => 2) # => https://sushi.com/api/nigiri?token=abc&page=2 - # - def build_url(url = nil, extra_params = nil) - uri = build_exclusive_url(url) - - query_values = params.dup.merge_query(uri.query, options.params_encoder) - query_values.update extra_params if extra_params - uri.query = query_values.empty? ? nil : query_values.to_query(options.params_encoder) - - uri - end - - # Builds and runs the Faraday::Request. - # - # method - The Symbol HTTP method. - # url - The String or URI to access. - # body - The String body - # headers - Hash of unencoded HTTP header key/value pairs. - # - # Returns a Faraday::Response. - def run_request(method, url, body, headers) - if !METHODS.include?(method) - raise ArgumentError, "unknown http method: #{method}" - end - - request = build_request(method) do |req| - req.url(url) if url - req.headers.update(headers) if headers - req.body = body if body - yield(req) if block_given? - end - - builder.build_response(self, request) - end - - # Creates and configures the request object. - # - # Returns the new Request. - def build_request(method) - Request.create(method) do |req| - req.params = self.params.dup - req.headers = self.headers.dup - req.options = self.options.merge(:proxy => self.proxy) - yield(req) if block_given? - end - end - - # Internal: Build an absolute URL based on url_prefix. - # - # url - A String or URI-like object - # params - A Faraday::Utils::ParamsHash to replace the query values - # of the resulting url (default: nil). - # - # Returns the resulting URI instance. - def build_exclusive_url(url = nil, params = nil) - url = nil if url.respond_to?(:empty?) and url.empty? - base = url_prefix - if url and base.path and base.path !~ /\/$/ - base = base.dup - base.path = base.path + '/' # ensure trailing slash - end - uri = url ? base + url : base - uri.query = params.to_query(options.params_encoder) if params - uri.query = nil if uri.query and uri.query.empty? - uri - end - - # Internal: Creates a duplicate of this Faraday::Connection. - # - # Returns a Faraday::Connection. - def dup - self.class.new(build_exclusive_url, :headers => headers.dup, :params => params.dup, :builder => builder.dup, :ssl => ssl.dup) - end - - # Internal: Yields username and password extracted from a URI if they both exist. - def with_uri_credentials(uri) - if uri.user and uri.password - yield(Utils.unescape(uri.user), Utils.unescape(uri.password)) - end - end - - def set_authorization_header(header_type, *args) - header = Faraday::Request.lookup_middleware(header_type). - header(*args) - headers[Faraday::Request::Authorization::KEY] = header - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/error.rb b/aviator/lib/puppet/feature/faraday/error.rb deleted file mode 100644 index 17712309d..000000000 --- a/aviator/lib/puppet/feature/faraday/error.rb +++ /dev/null @@ -1,53 +0,0 @@ -module Faraday - class Error < StandardError; end - class MissingDependency < Error; end - - class ClientError < Error - attr_reader :response - - def initialize(ex, response = nil) - @wrapped_exception = nil - @response = response - - if ex.respond_to?(:backtrace) - super(ex.message) - @wrapped_exception = ex - elsif ex.respond_to?(:each_key) - super("the server responded with status #{ex[:status]}") - @response = ex - else - super(ex.to_s) - end - end - - def backtrace - if @wrapped_exception - @wrapped_exception.backtrace - else - super - end - end - - def inspect - %(#<#{self.class}>) - end - end - - class ConnectionFailed < ClientError; end - class ResourceNotFound < ClientError; end - class ParsingError < ClientError; end - - class TimeoutError < ClientError - def initialize(ex = nil) - super(ex || "timeout") - end - end - - class SSLError < ClientError - end - - [:MissingDependency, :ClientError, :ConnectionFailed, :ResourceNotFound, - :ParsingError, :TimeoutError, :SSLError].each do |const| - Error.const_set(const, Faraday.const_get(const)) - end -end diff --git a/aviator/lib/puppet/feature/faraday/middleware.rb b/aviator/lib/puppet/feature/faraday/middleware.rb deleted file mode 100644 index c45d51ade..000000000 --- a/aviator/lib/puppet/feature/faraday/middleware.rb +++ /dev/null @@ -1,37 +0,0 @@ -module Faraday - class Middleware - extend MiddlewareRegistry - - class << self - attr_accessor :load_error - private :load_error= - end - - self.load_error = nil - - # Executes a block which should try to require and reference dependent libraries - def self.dependency(lib = nil) - lib ? require(lib) : yield - rescue LoadError, NameError => error - self.load_error = error - end - - def self.new(*) - raise "missing dependency for #{self}: #{load_error.message}" unless loaded? - super - end - - def self.loaded? - load_error.nil? - end - - def self.inherited(subclass) - super - subclass.send(:load_error=, self.load_error) - end - - def initialize(app = nil) - @app = app - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/options.rb b/aviator/lib/puppet/feature/faraday/options.rb deleted file mode 100644 index c1b36f60f..000000000 --- a/aviator/lib/puppet/feature/faraday/options.rb +++ /dev/null @@ -1,350 +0,0 @@ -module Faraday - # Subclasses Struct with some special helpers for converting from a Hash to - # a Struct. - class Options < Struct - # Public - def self.from(value) - value ? new.update(value) : new - end - - # Public - def each - return to_enum(:each) unless block_given? - members.each do |key| - yield(key.to_sym, send(key)) - end - end - - # Public - def update(obj) - obj.each do |key, value| - if sub_options = self.class.options_for(key) - value = sub_options.from(value) if value - elsif Hash === value - hash = {} - value.each do |hash_key, hash_value| - hash[hash_key] = hash_value - end - value = hash - end - - self.send("#{key}=", value) unless value.nil? - end - self - end - - alias merge! update - - # Public - def delete(key) - value = send(key) - send("#{key}=", nil) - value - end - - # Public - def clear - members.each { |member| delete(member) } - end - - # Public - def merge(value) - dup.update(value) - end - - # Public - def fetch(key, *args) - unless symbolized_key_set.include?(key.to_sym) - key_setter = "#{key}=" - if args.size > 0 - send(key_setter, args.first) - elsif block_given? - send(key_setter, Proc.new.call(key)) - else - raise self.class.fetch_error_class, "key not found: #{key.inspect}" - end - end - send(key) - end - - # Public - def values_at(*keys) - keys.map { |key| send(key) } - end - - # Public - def keys - members.reject { |member| send(member).nil? } - end - - # Public - def empty? - keys.empty? - end - - # Public - def each_key - return to_enum(:each_key) unless block_given? - keys.each do |key| - yield(key) - end - end - - # Public - def key?(key) - keys.include?(key) - end - - alias has_key? key? - - # Public - def each_value - return to_enum(:each_value) unless block_given? - values.each do |value| - yield(value) - end - end - - # Public - def value?(value) - values.include?(value) - end - - alias has_value? value? - - # Public - def to_hash - hash = {} - members.each do |key| - value = send(key) - hash[key.to_sym] = value unless value.nil? - end - hash - end - - # Internal - def inspect - values = [] - members.each do |member| - value = send(member) - values << "#{member}=#{value.inspect}" if value - end - values = values.empty? ? ' (empty)' : (' ' << values.join(", ")) - - %(#<#{self.class}#{values}>) - end - - # Internal - def self.options(mapping) - attribute_options.update(mapping) - end - - # Internal - def self.options_for(key) - attribute_options[key] - end - - # Internal - def self.attribute_options - @attribute_options ||= {} - end - - def self.memoized(key) - memoized_attributes[key.to_sym] = Proc.new - class_eval <<-RUBY, __FILE__, __LINE__ + 1 - def #{key}() self[:#{key}]; end - RUBY - end - - def self.memoized_attributes - @memoized_attributes ||= {} - end - - def [](key) - key = key.to_sym - if method = self.class.memoized_attributes[key] - super(key) || (self[key] = instance_eval(&method)) - else - super - end - end - - def symbolized_key_set - @symbolized_key_set ||= Set.new(keys.map { |k| k.to_sym }) - end - - def self.inherited(subclass) - super - subclass.attribute_options.update(attribute_options) - subclass.memoized_attributes.update(memoized_attributes) - end - - def self.fetch_error_class - @fetch_error_class ||= if Object.const_defined?(:KeyError) - ::KeyError - else - ::IndexError - end - end - end - - class RequestOptions < Options.new(:params_encoder, :proxy, :bind, - :timeout, :open_timeout, :boundary, - :oauth) - - def []=(key, value) - if key && key.to_sym == :proxy - super(key, value ? ProxyOptions.from(value) : nil) - else - super(key, value) - end - end - end - - class SSLOptions < Options.new(:verify, :ca_file, :ca_path, :verify_mode, - :cert_store, :client_cert, :client_key, :certificate, :private_key, :verify_depth, :version) - - def verify? - verify != false - end - - def disable? - !verify? - end - end - - class ProxyOptions < Options.new(:uri, :user, :password) - extend Forwardable - def_delegators :uri, :scheme, :scheme=, :host, :host=, :port, :port=, :path, :path= - - def self.from(value) - case value - when String - value = {:uri => Utils.URI(value)} - when URI - value = {:uri => value} - when Hash, Options - if uri = value.delete(:uri) - value[:uri] = Utils.URI(uri) - end - end - super(value) - end - - memoized(:user) { uri.user && Utils.unescape(uri.user) } - memoized(:password) { uri.password && Utils.unescape(uri.password) } - end - - class ConnectionOptions < Options.new(:request, :proxy, :ssl, :builder, :url, - :parallel_manager, :params, :headers, :builder_class) - - options :request => RequestOptions, :ssl => SSLOptions - - memoized(:request) { self.class.options_for(:request).new } - - memoized(:ssl) { self.class.options_for(:ssl).new } - - memoized(:builder_class) { RackBuilder } - - def new_builder(block) - builder_class.new(&block) - end - end - - class Env < Options.new(:method, :body, :url, :request, :request_headers, - :ssl, :parallel_manager, :params, :response, :response_headers, :status) - - ContentLength = 'Content-Length'.freeze - StatusesWithoutBody = Set.new [204, 304] - SuccessfulStatuses = 200..299 - - # A Set of HTTP verbs that typically send a body. If no body is set for - # these requests, the Content-Length header is set to 0. - MethodsWithBodies = Set.new [:post, :put, :patch, :options] - - options :request => RequestOptions, - :request_headers => Utils::Headers, :response_headers => Utils::Headers - - extend Forwardable - - def_delegators :request, :params_encoder - - # Public - def [](key) - if in_member_set?(key) - super(key) - else - custom_members[key] - end - end - - # Public - def []=(key, value) - if in_member_set?(key) - super(key, value) - else - custom_members[key] = value - end - end - - # Public - def success? - SuccessfulStatuses.include?(status) - end - - # Public - def needs_body? - !body && MethodsWithBodies.include?(method) - end - - # Public - def clear_body - request_headers[ContentLength] = '0' - self.body = '' - end - - # Public - def parse_body? - !StatusesWithoutBody.include?(status) - end - - # Public - def parallel? - !!parallel_manager - end - - def inspect - attrs = [nil] - members.each do |mem| - if value = send(mem) - attrs << "@#{mem}=#{value.inspect}" - end - end - if !custom_members.empty? - attrs << "@custom=#{custom_members.inspect}" - end - %(#<#{self.class}#{attrs.join(" ")}>) - end - - # Internal - def custom_members - @custom_members ||= {} - end - - # Internal - if members.first.is_a?(Symbol) - def in_member_set?(key) - self.class.member_set.include?(key.to_sym) - end - else - def in_member_set?(key) - self.class.member_set.include?(key.to_s) - end - end - - # Internal - def self.member_set - @member_set ||= Set.new(members) - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/parameters.rb b/aviator/lib/puppet/feature/faraday/parameters.rb deleted file mode 100644 index 136c43bca..000000000 --- a/aviator/lib/puppet/feature/faraday/parameters.rb +++ /dev/null @@ -1,193 +0,0 @@ -module Faraday - module NestedParamsEncoder - ESCAPE_RE = /[^a-zA-Z0-9 .~_-]/ - - def self.escape(s) - return s.to_s.gsub(ESCAPE_RE) { - '%' + $&.unpack('H2' * $&.bytesize).join('%').upcase - }.tr(' ', '+') - end - - def self.unescape(s) - CGI.unescape(s.to_s) - end - - def self.encode(params) - return nil if params == nil - - if !params.is_a?(Array) - if !params.respond_to?(:to_hash) - raise TypeError, - "Can't convert #{params.class} into Hash." - end - params = params.to_hash - params = params.map do |key, value| - key = key.to_s if key.kind_of?(Symbol) - [key, value] - end - # Useful default for OAuth and caching. - # Only to be used for non-Array inputs. Arrays should preserve order. - params.sort! - end - - # Helper lambda - to_query = lambda do |parent, value| - if value.is_a?(Hash) - value = value.map do |key, val| - key = escape(key) - [key, val] - end - value.sort! - buffer = "" - value.each do |key, val| - new_parent = "#{parent}%5B#{key}%5D" - buffer << "#{to_query.call(new_parent, val)}&" - end - return buffer.chop - elsif value.is_a?(Array) - buffer = "" - value.each_with_index do |val, i| - new_parent = "#{parent}%5B%5D" - buffer << "#{to_query.call(new_parent, val)}&" - end - return buffer.chop - else - encoded_value = escape(value) - return "#{parent}=#{encoded_value}" - end - end - - # The params have form [['key1', 'value1'], ['key2', 'value2']]. - buffer = '' - params.each do |parent, value| - encoded_parent = escape(parent) - buffer << "#{to_query.call(encoded_parent, value)}&" - end - return buffer.chop - end - - def self.decode(query) - return nil if query == nil - # Recursive helper lambda - dehash = lambda do |hash| - hash.each do |(key, value)| - if value.kind_of?(Hash) - hash[key] = dehash.call(value) - end - end - # Numeric keys implies an array - if hash != {} && hash.keys.all? { |key| key =~ /^\d+$/ } - hash.sort.inject([]) do |accu, (_, value)| - accu << value; accu - end - else - hash - end - end - - empty_accumulator = {} - return ((query.split('&').map do |pair| - pair.split('=', 2) if pair && !pair.empty? - end).compact.inject(empty_accumulator.dup) do |accu, (key, value)| - key = unescape(key) - if value.kind_of?(String) - value = unescape(value.gsub(/\+/, ' ')) - end - - array_notation = !!(key =~ /\[\]$/) - subkeys = key.split(/[\[\]]+/) - current_hash = accu - for i in 0...(subkeys.size - 1) - subkey = subkeys[i] - current_hash[subkey] = {} unless current_hash[subkey] - current_hash = current_hash[subkey] - end - if array_notation - current_hash[subkeys.last] = [] unless current_hash[subkeys.last] - current_hash[subkeys.last] << value - else - current_hash[subkeys.last] = value - end - accu - end).inject(empty_accumulator.dup) do |accu, (key, value)| - accu[key] = value.kind_of?(Hash) ? dehash.call(value) : value - accu - end - end - end - - module FlatParamsEncoder - ESCAPE_RE = /[^a-zA-Z0-9 .~_-]/ - - def self.escape(s) - return s.to_s.gsub(ESCAPE_RE) { - '%' + $&.unpack('H2' * $&.bytesize).join('%').upcase - }.tr(' ', '+') - end - - def self.unescape(s) - CGI.unescape(s.to_s) - end - - def self.encode(params) - return nil if params == nil - - if !params.is_a?(Array) - if !params.respond_to?(:to_hash) - raise TypeError, - "Can't convert #{params.class} into Hash." - end - params = params.to_hash - params = params.map do |key, value| - key = key.to_s if key.kind_of?(Symbol) - [key, value] - end - # Useful default for OAuth and caching. - # Only to be used for non-Array inputs. Arrays should preserve order. - params.sort! - end - - # The params have form [['key1', 'value1'], ['key2', 'value2']]. - buffer = '' - params.each do |key, value| - encoded_key = escape(key) - value = value.to_s if value == true || value == false - if value == nil - buffer << "#{encoded_key}&" - elsif value.kind_of?(Array) - value.each do |sub_value| - encoded_value = escape(sub_value) - buffer << "#{encoded_key}=#{encoded_value}&" - end - else - encoded_value = escape(value) - buffer << "#{encoded_key}=#{encoded_value}&" - end - end - return buffer.chop - end - - def self.decode(query) - empty_accumulator = {} - return nil if query == nil - split_query = (query.split('&').map do |pair| - pair.split('=', 2) if pair && !pair.empty? - end).compact - return split_query.inject(empty_accumulator.dup) do |accu, pair| - pair[0] = unescape(pair[0]) - pair[1] = true if pair[1].nil? - if pair[1].respond_to?(:to_str) - pair[1] = unescape(pair[1].to_str.gsub(/\+/, " ")) - end - if accu[pair[0]].kind_of?(Array) - accu[pair[0]] << pair[1] - elsif accu[pair[0]] - accu[pair[0]] = [accu[pair[0]], pair[1]] - else - accu[pair[0]] = pair[1] - end - accu - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/rack_builder.rb b/aviator/lib/puppet/feature/faraday/rack_builder.rb deleted file mode 100644 index 204ce41d1..000000000 --- a/aviator/lib/puppet/feature/faraday/rack_builder.rb +++ /dev/null @@ -1,212 +0,0 @@ -module Faraday - # A Builder that processes requests into responses by passing through an inner - # middleware stack (heavily inspired by Rack). - # - # Faraday::Connection.new(:url => 'http://sushi.com') do |builder| - # builder.request :url_encoded # Faraday::Request::UrlEncoded - # builder.adapter :net_http # Faraday::Adapter::NetHttp - # end - class RackBuilder - attr_accessor :handlers - - # Error raised when trying to modify the stack after calling `lock!` - class StackLocked < RuntimeError; end - - # borrowed from ActiveSupport::Dependencies::Reference & - # ActionDispatch::MiddlewareStack::Middleware - class Handler - @@constants_mutex = Mutex.new - @@constants = Hash.new { |h, k| - value = k.respond_to?(:constantize) ? k.constantize : Object.const_get(k) - @@constants_mutex.synchronize { h[k] = value } - } - - attr_reader :name - - def initialize(klass, *args, &block) - @name = klass.to_s - if klass.respond_to?(:name) - @@constants_mutex.synchronize { @@constants[@name] = klass } - end - @args, @block = args, block - end - - def klass() @@constants[@name] end - def inspect() @name end - - def ==(other) - if other.is_a? Handler - self.name == other.name - elsif other.respond_to? :name - klass == other - else - @name == other.to_s - end - end - - def build(app) - klass.new(app, *@args, &@block) - end - end - - def initialize(handlers = []) - @handlers = handlers - if block_given? - build(&Proc.new) - elsif @handlers.empty? - # default stack, if nothing else is configured - self.request :url_encoded - self.adapter Faraday.default_adapter - end - end - - def build(options = {}) - raise_if_locked - @handlers.clear unless options[:keep] - yield(self) if block_given? - end - - def [](idx) - @handlers[idx] - end - - # Locks the middleware stack to ensure no further modifications are possible. - def lock! - @handlers.freeze - end - - def locked? - @handlers.frozen? - end - - def use(klass, *args, &block) - if klass.is_a? Symbol - use_symbol(Faraday::Middleware, klass, *args, &block) - else - raise_if_locked - @handlers << self.class::Handler.new(klass, *args, &block) - end - end - - def request(key, *args, &block) - use_symbol(Faraday::Request, key, *args, &block) - end - - def response(key, *args, &block) - use_symbol(Faraday::Response, key, *args, &block) - end - - def adapter(key, *args, &block) - use_symbol(Faraday::Adapter, key, *args, &block) - end - - ## methods to push onto the various positions in the stack: - - def insert(index, *args, &block) - raise_if_locked - index = assert_index(index) - handler = self.class::Handler.new(*args, &block) - @handlers.insert(index, handler) - end - - alias_method :insert_before, :insert - - def insert_after(index, *args, &block) - index = assert_index(index) - insert(index + 1, *args, &block) - end - - def swap(index, *args, &block) - raise_if_locked - index = assert_index(index) - @handlers.delete_at(index) - insert(index, *args, &block) - end - - def delete(handler) - raise_if_locked - @handlers.delete(handler) - end - - # Processes a Request into a Response by passing it through this Builder's - # middleware stack. - # - # connection - Faraday::Connection - # request - Faraday::Request - # - # Returns a Faraday::Response. - def build_response(connection, request) - app.call(build_env(connection, request)) - end - - # The "rack app" wrapped in middleware. All requests are sent here. - # - # The builder is responsible for creating the app object. After this, - # the builder gets locked to ensure no further modifications are made - # to the middleware stack. - # - # Returns an object that responds to `call` and returns a Response. - def app - @app ||= begin - lock! - to_app(lambda { |env| - response = Response.new - response.finish(env) unless env.parallel? - env.response = response - }) - end - end - - def to_app(inner_app) - # last added handler is the deepest and thus closest to the inner app - @handlers.reverse.inject(inner_app) { |app, handler| handler.build(app) } - end - - def ==(other) - other.is_a?(self.class) && @handlers == other.handlers - end - - def dup - self.class.new(@handlers.dup) - end - - # ENV Keys - # :method - a symbolized request method (:get, :post) - # :body - the request body that will eventually be converted to a string. - # :url - URI instance for the current request. - # :status - HTTP response status code - # :request_headers - hash of HTTP Headers to be sent to the server - # :response_headers - Hash of HTTP headers from the server - # :parallel_manager - sent if the connection is in parallel mode - # :request - Hash of options for configuring the request. - # :timeout - open/read timeout Integer in seconds - # :open_timeout - read timeout Integer in seconds - # :proxy - Hash of proxy options - # :uri - Proxy Server URI - # :user - Proxy server username - # :password - Proxy server password - # :ssl - Hash of options for configuring SSL requests. - def build_env(connection, request) - Env.new(request.method, request.body, - connection.build_exclusive_url(request.path, request.params), - request.options, request.headers, connection.ssl, - connection.parallel_manager) - end - - private - - def raise_if_locked - raise StackLocked, "can't modify middleware stack after making a request" if locked? - end - - def use_symbol(mod, key, *args, &block) - use(mod.lookup_middleware(key), *args, &block) - end - - def assert_index(index) - idx = index.is_a?(Integer) ? index : @handlers.index(index) - raise "No such handler: #{index.inspect}" unless idx - idx - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/request.rb b/aviator/lib/puppet/feature/faraday/request.rb deleted file mode 100644 index 481077f14..000000000 --- a/aviator/lib/puppet/feature/faraday/request.rb +++ /dev/null @@ -1,92 +0,0 @@ -module Faraday - # Used to setup urls, params, headers, and the request body in a sane manner. - # - # @connection.post do |req| - # req.url 'http://localhost', 'a' => '1' # 'http://localhost?a=1' - # req.headers['b'] = '2' # Header - # req.params['c'] = '3' # GET Param - # req['b'] = '2' # also Header - # req.body = 'abc' - # end - # - class Request < Struct.new(:method, :path, :params, :headers, :body, :options) - extend MiddlewareRegistry - - register_middleware File.expand_path('../request', __FILE__), - :url_encoded => [:UrlEncoded, 'url_encoded'], - :multipart => [:Multipart, 'multipart'], - :retry => [:Retry, 'retry'], - :authorization => [:Authorization, 'authorization'], - :basic_auth => [:BasicAuthentication, 'basic_authentication'], - :token_auth => [:TokenAuthentication, 'token_authentication'], - :instrumentation => [:Instrumentation, 'instrumentation'] - - def self.create(request_method) - new(request_method).tap do |request| - yield(request) if block_given? - end - end - - # Public: Replace params, preserving the existing hash type - def params=(hash) - if params - params.replace hash - else - super - end - end - - # Public: Replace request headers, preserving the existing hash type - def headers=(hash) - if headers - headers.replace hash - else - super - end - end - - def url(path, params = nil) - if path.respond_to? :query - if query = path.query - path = path.dup - path.query = nil - end - else - path, query = path.split('?', 2) - end - self.path = path - self.params.merge_query query, options.params_encoder - self.params.update(params) if params - end - - def [](key) - headers[key] - end - - def []=(key, value) - headers[key] = value - end - - # ENV Keys - # :method - a symbolized request method (:get, :post) - # :body - the request body that will eventually be converted to a string. - # :url - URI instance for the current request. - # :status - HTTP response status code - # :request_headers - hash of HTTP Headers to be sent to the server - # :response_headers - Hash of HTTP headers from the server - # :parallel_manager - sent if the connection is in parallel mode - # :request - Hash of options for configuring the request. - # :timeout - open/read timeout Integer in seconds - # :open_timeout - read timeout Integer in seconds - # :proxy - Hash of proxy options - # :uri - Proxy Server URI - # :user - Proxy server username - # :password - Proxy server password - # :ssl - Hash of options for configuring SSL requests. - def to_env(connection) - Env.new(method, body, connection.build_exclusive_url(path, params), - options, headers, connection.ssl, connection.parallel_manager) - end - end -end - diff --git a/aviator/lib/puppet/feature/faraday/request/authorization.rb b/aviator/lib/puppet/feature/faraday/request/authorization.rb deleted file mode 100644 index 43b452880..000000000 --- a/aviator/lib/puppet/feature/faraday/request/authorization.rb +++ /dev/null @@ -1,42 +0,0 @@ -module Faraday - class Request::Authorization < Faraday::Middleware - KEY = "Authorization".freeze unless defined? KEY - - # Public - def self.header(type, token) - case token - when String, Symbol - "#{type} #{token}" - when Hash - build_hash(type.to_s, token) - else - raise ArgumentError, "Can't build an Authorization #{type} header from #{token.inspect}" - end - end - - # Internal - def self.build_hash(type, hash) - offset = KEY.size + type.size + 3 - comma = ",\n#{' ' * offset}" - values = [] - hash.each do |key, value| - values << "#{key}=#{value.to_s.inspect}" - end - "#{type} #{values * comma}" - end - - def initialize(app, type, token) - @header_value = self.class.header(type, token) - super(app) - end - - # Public - def call(env) - unless env.request_headers[KEY] - env.request_headers[KEY] = @header_value - end - @app.call(env) - end - end -end - diff --git a/aviator/lib/puppet/feature/faraday/request/basic_authentication.rb b/aviator/lib/puppet/feature/faraday/request/basic_authentication.rb deleted file mode 100644 index 54c8dee97..000000000 --- a/aviator/lib/puppet/feature/faraday/request/basic_authentication.rb +++ /dev/null @@ -1,13 +0,0 @@ -require 'base64' - -module Faraday - class Request::BasicAuthentication < Request.load_middleware(:authorization) - # Public - def self.header(login, pass) - value = Base64.encode64([login, pass].join(':')) - value.gsub!("\n", '') - super(:Basic, value) - end - end -end - diff --git a/aviator/lib/puppet/feature/faraday/request/instrumentation.rb b/aviator/lib/puppet/feature/faraday/request/instrumentation.rb deleted file mode 100644 index 42af8bc44..000000000 --- a/aviator/lib/puppet/feature/faraday/request/instrumentation.rb +++ /dev/null @@ -1,36 +0,0 @@ -module Faraday - class Request::Instrumentation < Faraday::Middleware - class Options < Faraday::Options.new(:name, :instrumenter) - def name - self[:name] ||= 'request.faraday' - end - - def instrumenter - self[:instrumenter] ||= ActiveSupport::Notifications - end - end - - # Public: Instruments requests using Active Support. - # - # Measures time spent only for synchronous requests. - # - # Examples - # - # ActiveSupport::Notifications.subscribe('request.faraday') do |name, starts, ends, _, env| - # url = env[:url] - # http_method = env[:method].to_s.upcase - # duration = ends - starts - # $stderr.puts '[%s] %s %s (%.3f s)' % [url.host, http_method, url.request_uri, duration] - # end - def initialize(app, options = nil) - super(app) - @name, @instrumenter = Options.from(options).values_at(:name, :instrumenter) - end - - def call(env) - @instrumenter.instrument(@name, env) do - @app.call(env) - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/request/multipart.rb b/aviator/lib/puppet/feature/faraday/request/multipart.rb deleted file mode 100644 index 38b452af7..000000000 --- a/aviator/lib/puppet/feature/faraday/request/multipart.rb +++ /dev/null @@ -1,63 +0,0 @@ -require File.expand_path("../url_encoded", __FILE__) - -module Faraday - class Request::Multipart < Request::UrlEncoded - self.mime_type = 'multipart/form-data'.freeze - DEFAULT_BOUNDARY = "-----------RubyMultipartPost".freeze unless defined? DEFAULT_BOUNDARY - - def call(env) - match_content_type(env) do |params| - env.request.boundary ||= DEFAULT_BOUNDARY - env.request_headers[CONTENT_TYPE] += "; boundary=#{env.request.boundary}" - env.body = create_multipart(env, params) - end - @app.call env - end - - def process_request?(env) - type = request_type(env) - env.body.respond_to?(:each_key) and !env.body.empty? and ( - (type.empty? and has_multipart?(env.body)) or - type == self.class.mime_type - ) - end - - def has_multipart?(obj) - # string is an enum in 1.8, returning list of itself - if obj.respond_to?(:each) && !obj.is_a?(String) - (obj.respond_to?(:values) ? obj.values : obj).each do |val| - return true if (val.respond_to?(:content_type) || has_multipart?(val)) - end - end - false - end - - def create_multipart(env, params) - boundary = env.request.boundary - parts = process_params(params) do |key, value| - Faraday::Parts::Part.new(boundary, key, value) - end - parts << Faraday::Parts::EpiloguePart.new(boundary) - - body = Faraday::CompositeReadIO.new(parts) - env.request_headers[Faraday::Env::ContentLength] = body.length.to_s - return body - end - - def process_params(params, prefix = nil, pieces = nil, &block) - params.inject(pieces || []) do |all, (key, value)| - key = "#{prefix}[#{key}]" if prefix - - case value - when Array - values = value.inject([]) { |a,v| a << [nil, v] } - process_params(values, key, all, &block) - when Hash - process_params(value, key, all, &block) - else - all << block.call(key, value) - end - end - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/request/retry.rb b/aviator/lib/puppet/feature/faraday/request/retry.rb deleted file mode 100644 index 08bc83766..000000000 --- a/aviator/lib/puppet/feature/faraday/request/retry.rb +++ /dev/null @@ -1,140 +0,0 @@ -module Faraday - # Catches exceptions and retries each request a limited number of times. - # - # By default, it retries 2 times and handles only timeout exceptions. It can - # be configured with an arbitrary number of retries, a list of exceptions to - # handle, a retry interval, a percentage of randomness to add to the retry - # interval, and a backoff factor. - # - # Examples - # - # Faraday.new do |conn| - # conn.request :retry, max: 2, interval: 0.05, - # interval_randomness: 0.5, backoff_factor: 2 - # exceptions: [CustomException, 'Timeout::Error'] - # conn.adapter ... - # end - # - # This example will result in a first interval that is random between 0.05 and 0.075 and a second - # interval that is random between 0.1 and 0.15 - # - class Request::Retry < Faraday::Middleware - - IDEMPOTENT_METHODS = [:delete, :get, :head, :options, :put] - - class Options < Faraday::Options.new(:max, :interval, :interval_randomness, :backoff_factor, :exceptions, :retry_if) - DEFAULT_CHECK = lambda { |env,exception| false } - - def self.from(value) - if Fixnum === value - new(value) - else - super(value) - end - end - - def max - (self[:max] ||= 2).to_i - end - - def interval - (self[:interval] ||= 0).to_f - end - - def interval_randomness - (self[:interval_randomness] ||= 0).to_i - end - - def backoff_factor - (self[:backoff_factor] ||= 1).to_f - end - - def exceptions - Array(self[:exceptions] ||= [Errno::ETIMEDOUT, 'Timeout::Error', - Error::TimeoutError]) - end - - def retry_if - self[:retry_if] ||= DEFAULT_CHECK - end - - end - - # Public: Initialize middleware - # - # Options: - # max - Maximum number of retries (default: 2) - # interval - Pause in seconds between retries (default: 0) - # interval_randomness - The maximum random interval amount expressed - # as a float between 0 and 1 to use in addition to the - # interval. (default: 0) - # backoff_factor - The amount to multiple each successive retry's - # interval amount by in order to provide backoff - # (default: 1) - # exceptions - The list of exceptions to handle. Exceptions can be - # given as Class, Module, or String. (default: - # [Errno::ETIMEDOUT, Timeout::Error, - # Error::TimeoutError]) - # retry_if - block that will receive the env object and the exception raised - # and should decide if the code should retry still the action or - # not independent of the retry count. This would be useful - # if the exception produced is non-recoverable or if the - # the HTTP method called is not idempotent. - # (defaults to return false) - def initialize(app, options = nil) - super(app) - @options = Options.from(options) - @errmatch = build_exception_matcher(@options.exceptions) - end - - def sleep_amount(retries) - retry_index = @options.max - retries - current_interval = @options.interval * (@options.backoff_factor ** retry_index) - random_interval = rand * @options.interval_randomness.to_f * @options.interval - current_interval + random_interval - end - - def call(env) - retries = @options.max - request_body = env[:body] - begin - env[:body] = request_body # after failure env[:body] is set to the response body - @app.call(env) - rescue @errmatch => exception - if retries > 0 && retry_request?(env, exception) - retries -= 1 - sleep sleep_amount(retries + 1) - retry - end - raise - end - end - - # Private: construct an exception matcher object. - # - # An exception matcher for the rescue clause can usually be any object that - # responds to `===`, but for Ruby 1.8 it has to be a Class or Module. - def build_exception_matcher(exceptions) - matcher = Module.new - (class << matcher; self; end).class_eval do - define_method(:===) do |error| - exceptions.any? do |ex| - if ex.is_a? Module - error.is_a? ex - else - error.class.to_s == ex.to_s - end - end - end - end - matcher - end - - private - - def retry_request?(env, exception) - IDEMPOTENT_METHODS.include?(env[:method]) || @options.retry_if.call(env, exception) - end - - end -end diff --git a/aviator/lib/puppet/feature/faraday/request/token_authentication.rb b/aviator/lib/puppet/feature/faraday/request/token_authentication.rb deleted file mode 100644 index 25586080c..000000000 --- a/aviator/lib/puppet/feature/faraday/request/token_authentication.rb +++ /dev/null @@ -1,15 +0,0 @@ -module Faraday - class Request::TokenAuthentication < Request.load_middleware(:authorization) - # Public - def self.header(token, options = nil) - options ||= {} - options[:token] = token - super(:Token, options) - end - - def initialize(app, token, options = nil) - super(app, token, options) - end - end -end - diff --git a/aviator/lib/puppet/feature/faraday/request/url_encoded.rb b/aviator/lib/puppet/feature/faraday/request/url_encoded.rb deleted file mode 100644 index b02a26621..000000000 --- a/aviator/lib/puppet/feature/faraday/request/url_encoded.rb +++ /dev/null @@ -1,36 +0,0 @@ -module Faraday - class Request::UrlEncoded < Faraday::Middleware - CONTENT_TYPE = 'Content-Type'.freeze unless defined? CONTENT_TYPE - - class << self - attr_accessor :mime_type - end - self.mime_type = 'application/x-www-form-urlencoded'.freeze - - def call(env) - match_content_type(env) do |data| - params = Faraday::Utils::ParamsHash[data] - env.body = params.to_query(env.params_encoder) - end - @app.call env - end - - def match_content_type(env) - if process_request?(env) - env.request_headers[CONTENT_TYPE] ||= self.class.mime_type - yield(env.body) unless env.body.respond_to?(:to_str) - end - end - - def process_request?(env) - type = request_type(env) - env.body and (type.empty? or type == self.class.mime_type) - end - - def request_type(env) - type = env.request_headers[CONTENT_TYPE].to_s - type = type.split(';', 2).first if type.index(';') - type - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/response.rb b/aviator/lib/puppet/feature/faraday/response.rb deleted file mode 100644 index 88ec5310a..000000000 --- a/aviator/lib/puppet/feature/faraday/response.rb +++ /dev/null @@ -1,93 +0,0 @@ -require 'forwardable' - -module Faraday - class Response - # Used for simple response middleware. - class Middleware < Faraday::Middleware - def call(env) - @app.call(env).on_complete do |environment| - on_complete(environment) - end - end - - # Override this to modify the environment after the response has finished. - # Calls the `parse` method if defined - def on_complete(env) - env.body = parse(env.body) if respond_to?(:parse) && env.parse_body? - end - end - - extend Forwardable - extend MiddlewareRegistry - - register_middleware File.expand_path('../response', __FILE__), - :raise_error => [:RaiseError, 'raise_error'], - :logger => [:Logger, 'logger'] - - def initialize(env = nil) - @env = Env.from(env) if env - @on_complete_callbacks = [] - end - - attr_reader :env - - def_delegators :env, :to_hash - - def status - finished? ? env.status : nil - end - - def headers - finished? ? env.response_headers : {} - end - def_delegator :headers, :[] - - def body - finished? ? env.body : nil - end - - def finished? - !!env - end - - def on_complete - if not finished? - @on_complete_callbacks << Proc.new - else - yield(env) - end - return self - end - - def finish(env) - raise "response already finished" if finished? - @on_complete_callbacks.each { |callback| callback.call(env) } - @env = Env.from(env) - return self - end - - def success? - finished? && env.success? - end - - # because @on_complete_callbacks cannot be marshalled - def marshal_dump - !finished? ? nil : { - :status => @env.status, :body => @env.body, - :response_headers => @env.response_headers - } - end - - def marshal_load(env) - @env = Env.from(env) - end - - # Expand the env with more properties, without overriding existing ones. - # Useful for applying request params after restoring a marshalled Response. - def apply_request(request_env) - raise "response didn't finish yet" unless finished? - @env = Env.from(request_env).update(@env) - return self - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/response/logger.rb b/aviator/lib/puppet/feature/faraday/response/logger.rb deleted file mode 100644 index cab7f1b7c..000000000 --- a/aviator/lib/puppet/feature/faraday/response/logger.rb +++ /dev/null @@ -1,34 +0,0 @@ -require 'forwardable' - -module Faraday - class Response::Logger < Response::Middleware - extend Forwardable - - def initialize(app, logger = nil) - super(app) - @logger = logger || begin - require 'logger' - ::Logger.new(STDOUT) - end - end - - def_delegators :@logger, :debug, :info, :warn, :error, :fatal - - def call(env) - info "#{env.method} #{env.url.to_s}" - debug('request') { dump_headers env.request_headers } - super - end - - def on_complete(env) - info('Status') { env.status.to_s } - debug('response') { dump_headers env.response_headers } - end - - private - - def dump_headers(headers) - headers.map { |k, v| "#{k}: #{v.inspect}" }.join("\n") - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/response/raise_error.rb b/aviator/lib/puppet/feature/faraday/response/raise_error.rb deleted file mode 100644 index 437762bc1..000000000 --- a/aviator/lib/puppet/feature/faraday/response/raise_error.rb +++ /dev/null @@ -1,21 +0,0 @@ -module Faraday - class Response::RaiseError < Response::Middleware - ClientErrorStatuses = 400...600 - - def on_complete(env) - case env[:status] - when 404 - raise Faraday::Error::ResourceNotFound, response_values(env) - when 407 - # mimic the behavior that we get with proxy requests with HTTPS - raise Faraday::Error::ConnectionFailed, %{407 "Proxy Authentication Required "} - when ClientErrorStatuses - raise Faraday::Error::ClientError, response_values(env) - end - end - - def response_values(env) - {:status => env.status, :headers => env.response_headers, :body => env.body} - end - end -end diff --git a/aviator/lib/puppet/feature/faraday/upload_io.rb b/aviator/lib/puppet/feature/faraday/upload_io.rb deleted file mode 100644 index 9130d159d..000000000 --- a/aviator/lib/puppet/feature/faraday/upload_io.rb +++ /dev/null @@ -1,67 +0,0 @@ -begin - require 'composite_io' - require 'parts' - require 'stringio' -rescue LoadError - $stderr.puts "Install the multipart-post gem." - raise -end - -module Faraday - # Similar but not compatible with ::CompositeReadIO provided by multipart-post. - class CompositeReadIO - def initialize(*parts) - @parts = parts.flatten - @ios = @parts.map { |part| part.to_io } - @index = 0 - end - - def length - @parts.inject(0) { |sum, part| sum + part.length } - end - - def rewind - @ios.each { |io| io.rewind } - @index = 0 - end - - # Read from IOs in order until `length` bytes have been received. - def read(length = nil, outbuf = nil) - got_result = false - outbuf = outbuf ? outbuf.replace("") : "" - - while io = current_io - if result = io.read(length) - got_result ||= !result.nil? - result.force_encoding("BINARY") if result.respond_to?(:force_encoding) - outbuf << result - length -= result.length if length - break if length == 0 - end - advance_io - end - (!got_result && length) ? nil : outbuf - end - - def close - @ios.each { |io| io.close } - end - - def ensure_open_and_readable - # Rubinius compatibility - end - - private - - def current_io - @ios[@index] - end - - def advance_io - @index += 1 - end - end - - UploadIO = ::UploadIO - Parts = ::Parts -end diff --git a/aviator/lib/puppet/feature/faraday/utils.rb b/aviator/lib/puppet/feature/faraday/utils.rb deleted file mode 100644 index 1cd6526a6..000000000 --- a/aviator/lib/puppet/feature/faraday/utils.rb +++ /dev/null @@ -1,297 +0,0 @@ -require 'thread' -Faraday.require_libs 'parameters' - -module Faraday - module Utils - extend self - - # Adapted from Rack::Utils::HeaderHash - class Headers < ::Hash - def self.from(value) - new(value) - end - - def initialize(hash = nil) - super() - @names = {} - self.update(hash || {}) - end - - # need to synchronize concurrent writes to the shared KeyMap - keymap_mutex = Mutex.new - - # symbol -> string mapper + cache - KeyMap = Hash.new do |map, key| - value = if key.respond_to?(:to_str) - key - else - key.to_s.split('_'). # :user_agent => %w(user agent) - each { |w| w.capitalize! }. # => %w(User Agent) - join('-') # => "User-Agent" - end - keymap_mutex.synchronize { map[key] = value } - end - KeyMap[:etag] = "ETag" - - def [](k) - k = KeyMap[k] - super(k) || super(@names[k.downcase]) - end - - def []=(k, v) - k = KeyMap[k] - k = (@names[k.downcase] ||= k) - # join multiple values with a comma - v = v.to_ary.join(', ') if v.respond_to? :to_ary - super(k, v) - end - - def fetch(k, *args, &block) - k = KeyMap[k] - key = @names.fetch(k.downcase, k) - super(key, *args, &block) - end - - def delete(k) - k = KeyMap[k] - if k = @names[k.downcase] - @names.delete k.downcase - super(k) - end - end - - def include?(k) - @names.include? k.downcase - end - - alias_method :has_key?, :include? - alias_method :member?, :include? - alias_method :key?, :include? - - def merge!(other) - other.each { |k, v| self[k] = v } - self - end - alias_method :update, :merge! - - def merge(other) - hash = dup - hash.merge! other - end - - def replace(other) - clear - self.update other - self - end - - def to_hash() ::Hash.new.update(self) end - - def parse(header_string) - return unless header_string && !header_string.empty? - header_string.split(/\r\n/). - tap { |a| a.shift if a.first.index('HTTP/') == 0 }. # drop the HTTP status line - map { |h| h.split(/:\s+/, 2) }.reject { |p| p[0].nil? }. # split key and value, ignore blank lines - each { |key, value| - # join multiple values with a comma - if self[key] - self[key] << ', ' << value - else - self[key] = value - end - } - end - end - - # hash with stringified keys - class ParamsHash < Hash - def [](key) - super(convert_key(key)) - end - - def []=(key, value) - super(convert_key(key), value) - end - - def delete(key) - super(convert_key(key)) - end - - def include?(key) - super(convert_key(key)) - end - - alias_method :has_key?, :include? - alias_method :member?, :include? - alias_method :key?, :include? - - def update(params) - params.each do |key, value| - self[key] = value - end - self - end - alias_method :merge!, :update - - def merge(params) - dup.update(params) - end - - def replace(other) - clear - update(other) - end - - def merge_query(query, encoder = nil) - if query && !query.empty? - update((encoder || Utils.default_params_encoder).decode(query)) - end - self - end - - def to_query(encoder = nil) - (encoder || Utils.default_params_encoder).encode(self) - end - - private - - def convert_key(key) - key.to_s - end - end - - def build_query(params) - FlatParamsEncoder.encode(params) - end - - def build_nested_query(params) - NestedParamsEncoder.encode(params) - end - - ESCAPE_RE = /[^a-zA-Z0-9 .~_-]/ - - def escape(s) - s.to_s.gsub(ESCAPE_RE) {|match| - '%' + match.unpack('H2' * match.bytesize).join('%').upcase - }.tr(' ', '+') - end - - def unescape(s) CGI.unescape s.to_s end - - DEFAULT_SEP = /[&;] */n - - # Adapted from Rack - def parse_query(query) - FlatParamsEncoder.decode(query) - end - - def parse_nested_query(query) - NestedParamsEncoder.decode(query) - end - - def default_params_encoder - @default_params_encoder ||= NestedParamsEncoder - end - - class << self - attr_writer :default_params_encoder - end - - # Stolen from Rack - def normalize_params(params, name, v = nil) - name =~ %r(\A[\[\]]*([^\[\]]+)\]*) - k = $1 || '' - after = $' || '' - - return if k.empty? - - if after == "" - if params[k] - params[k] = Array[params[k]] unless params[k].kind_of?(Array) - params[k] << v - else - params[k] = v - end - elsif after == "[]" - params[k] ||= [] - raise TypeError, "expected Array (got #{params[k].class.name}) for param `#{k}'" unless params[k].is_a?(Array) - params[k] << v - elsif after =~ %r(^\[\]\[([^\[\]]+)\]$) || after =~ %r(^\[\](.+)$) - child_key = $1 - params[k] ||= [] - raise TypeError, "expected Array (got #{params[k].class.name}) for param `#{k}'" unless params[k].is_a?(Array) - if params[k].last.is_a?(Hash) && !params[k].last.key?(child_key) - normalize_params(params[k].last, child_key, v) - else - params[k] << normalize_params({}, child_key, v) - end - else - params[k] ||= {} - raise TypeError, "expected Hash (got #{params[k].class.name}) for param `#{k}'" unless params[k].is_a?(Hash) - params[k] = normalize_params(params[k], after, v) - end - - return params - end - - # Normalize URI() behavior across Ruby versions - # - # url - A String or URI. - # - # Returns a parsed URI. - def URI(url) - if url.respond_to?(:host) - url - elsif url.respond_to?(:to_str) - default_uri_parser.call(url) - else - raise ArgumentError, "bad argument (expected URI object or URI string)" - end - end - - def default_uri_parser - @default_uri_parser ||= begin - require 'uri' - Kernel.method(:URI) - end - end - - def default_uri_parser=(parser) - @default_uri_parser = if parser.respond_to?(:call) || parser.nil? - parser - else - parser.method(:parse) - end - end - - # Receives a String or URI and returns just the path with the query string sorted. - def normalize_path(url) - url = URI(url) - (url.path.start_with?('/') ? url.path : '/' + url.path) + - (url.query ? "?#{sort_query_params(url.query)}" : "") - end - - # Recursive hash update - def deep_merge!(target, hash) - hash.each do |key, value| - if Hash === value and Hash === target[key] - target[key] = deep_merge(target[key], value) - else - target[key] = value - end - end - target - end - - # Recursive hash merge - def deep_merge(source, hash) - deep_merge!(source.dup, hash) - end - - protected - - def sort_query_params(query) - query.split('&').sort.join('&') - end - end -end diff --git a/aviator/lib/puppet/feature/multipart_post.rb b/aviator/lib/puppet/feature/multipart_post.rb deleted file mode 100644 index 76540a8b7..000000000 --- a/aviator/lib/puppet/feature/multipart_post.rb +++ /dev/null @@ -1,9 +0,0 @@ -#-- -# Copyright (c) 2007-2013 Nick Sieger. -# See the file README.txt included with the distribution for -# software license details. -#++ - -module MultipartPost - VERSION = "2.0.0" -end diff --git a/aviator/lib/puppet/feature/multipartable.rb b/aviator/lib/puppet/feature/multipartable.rb deleted file mode 100644 index 28fa41e6d..000000000 --- a/aviator/lib/puppet/feature/multipartable.rb +++ /dev/null @@ -1,29 +0,0 @@ -#-- -# Copyright (c) 2007-2013 Nick Sieger. -# See the file README.txt included with the distribution for -# software license details. -#++ - -require 'parts' - module Multipartable - DEFAULT_BOUNDARY = "-----------RubyMultipartPost" - def initialize(path, params, headers={}, boundary = DEFAULT_BOUNDARY) - headers = headers.clone # don't want to modify the original variable - parts_headers = headers.delete(:parts) || {} - super(path, headers) - parts = params.map do |k,v| - case v - when Array - v.map {|item| Parts::Part.new(boundary, k, item, parts_headers[k]) } - else - Parts::Part.new(boundary, k, v, parts_headers[k]) - end - end.flatten - parts << Parts::EpiloguePart.new(boundary) - ios = parts.map {|p| p.to_io } - self.set_content_type(headers["Content-Type"] || "multipart/form-data", - { "boundary" => boundary }) - self.content_length = parts.inject(0) {|sum,i| sum + i.length } - self.body_stream = CompositeReadIO.new(*ios) - end - end diff --git a/aviator/lib/puppet/feature/net/http/post/multipart.rb b/aviator/lib/puppet/feature/net/http/post/multipart.rb deleted file mode 100644 index 757058220..000000000 --- a/aviator/lib/puppet/feature/net/http/post/multipart.rb +++ /dev/null @@ -1,27 +0,0 @@ -#-- -# Copyright (c) 2007-2012 Nick Sieger. -# See the file README.txt included with the distribution for -# software license details. -#++ - -require 'net/http' -require 'stringio' -require 'cgi' -require 'composite_io' -require 'multipartable' -require 'parts' - -module Net #:nodoc: - class HTTP #:nodoc: - class Put - class Multipart < Put - include Multipartable - end - end - class Post #:nodoc: - class Multipart < Post - include Multipartable - end - end - end -end diff --git a/aviator/lib/puppet/feature/parts.rb b/aviator/lib/puppet/feature/parts.rb deleted file mode 100644 index c06cbd95d..000000000 --- a/aviator/lib/puppet/feature/parts.rb +++ /dev/null @@ -1,96 +0,0 @@ -#-- -# Copyright (c) 2007-2013 Nick Sieger. -# See the file README.txt included with the distribution for -# software license details. -#++ - -module Parts - module Part #:nodoc: - def self.new(boundary, name, value, headers = {}) - headers ||= {} # avoid nil values - if file?(value) - FilePart.new(boundary, name, value, headers) - else - ParamPart.new(boundary, name, value, headers) - end - end - - def self.file?(value) - value.respond_to?(:content_type) && value.respond_to?(:original_filename) - end - - def length - @part.length - end - - def to_io - @io - end - end - - class ParamPart - include Part - def initialize(boundary, name, value, headers = {}) - @part = build_part(boundary, name, value, headers) - @io = StringIO.new(@part) - end - - def length - @part.bytesize - end - - def build_part(boundary, name, value, headers = {}) - part = '' - part << "--#{boundary}\r\n" - part << "Content-Disposition: form-data; name=\"#{name.to_s}\"\r\n" - part << "Content-Type: #{headers["Content-Type"]}\r\n" if headers["Content-Type"] - part << "\r\n" - part << "#{value}\r\n" - end - end - - # Represents a part to be filled from file IO. - class FilePart - include Part - attr_reader :length - def initialize(boundary, name, io, headers = {}) - file_length = io.respond_to?(:length) ? io.length : File.size(io.local_path) - @head = build_head(boundary, name, io.original_filename, io.content_type, file_length, - io.respond_to?(:opts) ? io.opts.merge(headers) : headers) - @foot = "\r\n" - @length = @head.bytesize + file_length + @foot.length - @io = CompositeReadIO.new(StringIO.new(@head), io, StringIO.new(@foot)) - end - - def build_head(boundary, name, filename, type, content_len, opts = {}, headers = {}) - trans_encoding = opts["Content-Transfer-Encoding"] || "binary" - content_disposition = opts["Content-Disposition"] || "form-data" - - part = '' - part << "--#{boundary}\r\n" - part << "Content-Disposition: #{content_disposition}; name=\"#{name.to_s}\"; filename=\"#{filename}\"\r\n" - part << "Content-Length: #{content_len}\r\n" - if content_id = opts["Content-ID"] - part << "Content-ID: #{content_id}\r\n" - end - - if headers["Content-Type"] != nil - part << "Content-Type: " + headers["Content-Type"] + "\r\n" - else - part << "Content-Type: #{type}\r\n" - end - - part << "Content-Transfer-Encoding: #{trans_encoding}\r\n" - part << "\r\n" - end - end - - # Represents the epilogue or closing boundary. - class EpiloguePart - include Part - def initialize(boundary) - @part = "--#{boundary}--\r\n\r\n" - @io = StringIO.new(@part) - end - end -end From 425254ef6cbf7a6a0304422adf313e6dbf07d0f6 Mon Sep 17 00:00:00 2001 From: Gael Chamoulaud Date: Wed, 3 Feb 2016 19:22:35 +0100 Subject: [PATCH 22/35] Drop puppet-ipa-1 module Unused Puppet Module. Change-Id: I6bc4d09f08e3e3d3a1e4bb5d8925db8ab27485bf Signed-off-by: Gael Chamoulaud (cherry picked from commit f8da5b32372e53df91642b35ad94d51f12241976) --- Puppetfile | 4 - ipa/.fixtures.yml | 6 - ipa/.gitignore | 6 - ipa/.ruby-version | 1 - ipa/.travis.yml | 18 - ipa/Gemfile | 13 - ipa/Gemfile.lock | 43 -- ipa/Modulefile | 13 - ipa/README.md | 399 ------------------ ipa/Rakefile | 11 - ipa/lib/facter/ipa_adminhomedir.rb | 10 - ipa/lib/facter/ipa_adminuidnumber.rb | 10 - ipa/lib/facter/ipa_replicascheme.rb | 16 - .../parser/functions/ipa_string2hash.rb | 17 - ipa/manifests/adminconfig.pp | 64 --- ipa/manifests/cleanup.pp | 44 -- ipa/manifests/client.pp | 155 ------- ipa/manifests/clientinstall.pp | 49 --- ipa/manifests/configautomount.pp | 47 --- ipa/manifests/configsudo.pp | 62 --- ipa/manifests/createreplicas.pp | 6 - ipa/manifests/flushcache.pp | 16 - ipa/manifests/hostadd.pp | 21 - ipa/manifests/hostdelete.pp | 10 - ipa/manifests/init.pp | 350 --------------- ipa/manifests/loadbalanceconf.pp | 34 -- ipa/manifests/master.pp | 227 ---------- ipa/manifests/master_extca.pp | 103 ----- ipa/manifests/masterprincipal.pp | 18 - ipa/manifests/masterreplicationfirewall.pp | 13 - ipa/manifests/message.pp | 37 -- ipa/manifests/replica.pp | 74 ---- ipa/manifests/replicaagreement.pp | 10 - ipa/manifests/replicainstall.pp | 37 -- ipa/manifests/replicaprepare.pp | 30 -- ipa/manifests/replicapreparefirewall.pp | 13 - ipa/manifests/replicareplicationfirewall.pp | 13 - ipa/manifests/serverinstall.pp | 45 -- ipa/manifests/serverinstall_extca.pp | 29 -- ipa/manifests/service.pp | 13 - ipa/manifests/serviceadd.pp | 25 -- ipa/spec/classes/ipa_spec.rb | 24 -- ipa/spec/spec_helper.rb | 4 - ipa/templates/autofs_ldap_auth.conf.erb | 7 - ipa/templates/sudo-ldap.conf.erb | 12 - ipa/tests/init.pp | 11 - 46 files changed, 2170 deletions(-) delete mode 100644 ipa/.fixtures.yml delete mode 100644 ipa/.gitignore delete mode 100644 ipa/.ruby-version delete mode 100644 ipa/.travis.yml delete mode 100644 ipa/Gemfile delete mode 100644 ipa/Gemfile.lock delete mode 100644 ipa/Modulefile delete mode 100644 ipa/README.md delete mode 100644 ipa/Rakefile delete mode 100644 ipa/lib/facter/ipa_adminhomedir.rb delete mode 100644 ipa/lib/facter/ipa_adminuidnumber.rb delete mode 100644 ipa/lib/facter/ipa_replicascheme.rb delete mode 100644 ipa/lib/puppet/parser/functions/ipa_string2hash.rb delete mode 100644 ipa/manifests/adminconfig.pp delete mode 100644 ipa/manifests/cleanup.pp delete mode 100644 ipa/manifests/client.pp delete mode 100644 ipa/manifests/clientinstall.pp delete mode 100644 ipa/manifests/configautomount.pp delete mode 100644 ipa/manifests/configsudo.pp delete mode 100644 ipa/manifests/createreplicas.pp delete mode 100644 ipa/manifests/flushcache.pp delete mode 100644 ipa/manifests/hostadd.pp delete mode 100644 ipa/manifests/hostdelete.pp delete mode 100644 ipa/manifests/init.pp delete mode 100644 ipa/manifests/loadbalanceconf.pp delete mode 100644 ipa/manifests/master.pp delete mode 100644 ipa/manifests/master_extca.pp delete mode 100644 ipa/manifests/masterprincipal.pp delete mode 100644 ipa/manifests/masterreplicationfirewall.pp delete mode 100644 ipa/manifests/message.pp delete mode 100644 ipa/manifests/replica.pp delete mode 100644 ipa/manifests/replicaagreement.pp delete mode 100644 ipa/manifests/replicainstall.pp delete mode 100644 ipa/manifests/replicaprepare.pp delete mode 100644 ipa/manifests/replicapreparefirewall.pp delete mode 100644 ipa/manifests/replicareplicationfirewall.pp delete mode 100644 ipa/manifests/serverinstall.pp delete mode 100644 ipa/manifests/serverinstall_extca.pp delete mode 100644 ipa/manifests/service.pp delete mode 100644 ipa/manifests/serviceadd.pp delete mode 100644 ipa/spec/classes/ipa_spec.rb delete mode 100644 ipa/spec/spec_helper.rb delete mode 100644 ipa/templates/autofs_ldap_auth.conf.erb delete mode 100644 ipa/templates/sudo-ldap.conf.erb delete mode 100644 ipa/tests/init.pp diff --git a/Puppetfile b/Puppetfile index 39eb6f6ab..92d707ab0 100644 --- a/Puppetfile +++ b/Puppetfile @@ -94,10 +94,6 @@ mod 'inifile', :commit => '27857c28fc760a6010960226a4777f7b9bcf239e', :git => 'https://github.com/puppetlabs/puppetlabs-inifile.git' -mod 'ipa', - :commit => '4e634b31633bc956341e20980233107a6d5dc1f9', - :git => 'https://github.com/xbezdick/puppet-ipa-1.git' - mod 'ironic', :commit => '92ba5d6f7be0f1a24664888f4a8767ab3ab9e703', :git => 'https://github.com/openstack/puppet-ironic.git' diff --git a/ipa/.fixtures.yml b/ipa/.fixtures.yml deleted file mode 100644 index 6bf232cce..000000000 --- a/ipa/.fixtures.yml +++ /dev/null @@ -1,6 +0,0 @@ -fixtures: - repositories: - stdlib: git://github.com/puppetlabs/puppetlabs-stdlib.git - firewall: git://github.com/puppetlabs/puppetlabs-firewall.git - symlinks: - ipa: "#{source_dir}" diff --git a/ipa/.gitignore b/ipa/.gitignore deleted file mode 100644 index a75696b4d..000000000 --- a/ipa/.gitignore +++ /dev/null @@ -1,6 +0,0 @@ -.DS_Store -*.swp -*.bak -*~ -pkg -spec/fixtures/ diff --git a/ipa/.ruby-version b/ipa/.ruby-version deleted file mode 100644 index 4ff999b2c..000000000 --- a/ipa/.ruby-version +++ /dev/null @@ -1 +0,0 @@ -1.8.7-p358 diff --git a/ipa/.travis.yml b/ipa/.travis.yml deleted file mode 100644 index fc64c2aa6..000000000 --- a/ipa/.travis.yml +++ /dev/null @@ -1,18 +0,0 @@ -before_install: -- gem update --system 2.1.11 -- gem --version -rvm: 1.8.7 -bundler_args: '' -notifications: - email: - - tfhartmann@gmail.com - - robruma@gmail.com - hipchat: - rooms: - secure: iWj32dXbTmD2xxeUSXYgEHyA0bPtxS4Qz4O8XKED0nP6X+RTCpuYaXdeVEcFNEPMIbUN2KLJV+oHs4UgpA6dkMbpEqvz1qrVHPy7ex1+QwIvW/mJ8gJO7KQsRXLnM7pnRj5LzHOdoFv/Ou6dxIqdbObpnrABSOvcoK/hsKs45fE= -env: -- PUPPET_VERSION=3.3.1 -- PUPPET_VERSION=3.2.3 -- PUPPET_VERSION=2.7.23 -- PUPPET_VERSION=2.7.21 -- PUPPET_VERSION=2.7.17 diff --git a/ipa/Gemfile b/ipa/Gemfile deleted file mode 100644 index f102c37e8..000000000 --- a/ipa/Gemfile +++ /dev/null @@ -1,13 +0,0 @@ -source 'https://rubygems.org' - -if ENV.key?('PUPPET_VERSION') - puppetversion = "= #{ENV['PUPPET_VERSION']}" -else - puppetversion = ['>= 2.7'] -end - -gem 'rake' -gem 'puppet-lint' -gem 'rspec-puppet' -gem 'puppetlabs_spec_helper' -gem 'puppet', puppetversion diff --git a/ipa/Gemfile.lock b/ipa/Gemfile.lock deleted file mode 100644 index c04d17a43..000000000 --- a/ipa/Gemfile.lock +++ /dev/null @@ -1,43 +0,0 @@ -GEM - remote: https://rubygems.org/ - specs: - diff-lcs (1.2.4) - facter (1.7.3) - hiera (1.2.1) - json_pure - json_pure (1.8.1) - metaclass (0.0.1) - mocha (0.14.0) - metaclass (~> 0.0.1) - puppet (3.3.1) - facter (~> 1.6) - hiera (~> 1.0) - rgen (~> 0.6.5) - puppet-lint (0.3.2) - puppetlabs_spec_helper (0.4.1) - mocha (>= 0.10.5) - rake - rspec (>= 2.9.0) - rspec-puppet (>= 0.1.1) - rake (10.1.0) - rgen (0.6.6) - rspec (2.14.1) - rspec-core (~> 2.14.0) - rspec-expectations (~> 2.14.0) - rspec-mocks (~> 2.14.0) - rspec-core (2.14.6) - rspec-expectations (2.14.3) - diff-lcs (>= 1.1.3, < 2.0) - rspec-mocks (2.14.4) - rspec-puppet (0.1.6) - rspec - -PLATFORMS - ruby - -DEPENDENCIES - puppet (>= 2.7) - puppet-lint - puppetlabs_spec_helper - rake - rspec-puppet diff --git a/ipa/Modulefile b/ipa/Modulefile deleted file mode 100644 index 659741e1e..000000000 --- a/ipa/Modulefile +++ /dev/null @@ -1,13 +0,0 @@ -name 'huit-ipa' -version '1.5.5' -source 'Harvard University Information Technology' -author 'huit' -license 'Apache License, Version 2.0' -summary 'Manage IPA master, replicas, and clients' -description 'This module enables Puppet management of IPA masters, replicas, and clients. Master and replica support has been tested with RHEL6 and Red Hat IPA/FreeIPA 3.x; client support has also been tested with Ubuntu 10.04 LTS and Debian GNU/Linux 7.1 (wheezy).' -project_page 'https://github.com/huit/puppet-ipa' - -## Add dependencies, if any: -# dependency 'username/name', '>= 1.2.0' -dependency 'puppetlabs/firewall' -dependency 'puppetlabs/stdlib' diff --git a/ipa/README.md b/ipa/README.md deleted file mode 100644 index daf75daeb..000000000 --- a/ipa/README.md +++ /dev/null @@ -1,399 +0,0 @@ -# IPA Puppet module -[![Build Status](https://travis-ci.org/huit/puppet-ipa.png?branch=master)](https://travis-ci.org/huit/puppet-ipa) - -## Overview - -Puppet module that can manage an IPA master, replicas and clients. - -huit/puppet-ipa aims at the management and configuration of a complete IPA environment under Puppet control. - -To start, an IPA master will be required as the beginning of the LDAP/Kerberos environment. IPA replicas can -then be added for additional resiliancy. - -The IPA primary master server will take a minimum of two Puppet runs to fully configure. This is because the ipa_adminhomedir, -ipa_adminuidnumber and ipa_replicascheme facts are not available until the IPA primary master is installed and operational. -These facts are necessary to automatically configure clients and replicas. - -IPA replica servers will be automatically configured with a replication agreement on the IPA primary master server. - -All Puppet nodes added as clients will automatically be added to the IPA domain through exported resources. - -Multiple IPA domains are supported. - -A cleanup parameter has been included to remove the IPA server or client packages from nodes. - -## Dependencies - -The ability to use [Exported resources](http://docs.puppetlabs.com/guides/exported_resources.html) and -[Stored Configuration](http://projects.puppetlabs.com/projects/1/wiki/Using_Stored_Configuration) enabled on the Puppet master. - -[puppetlabs/puppetlabs-firewall](https://forge.puppetlabs.com/puppetlabs/firewall) module. - -[puppetlabs/stdlib](https://forge.puppetlabs.com/puppetlabs/stdlib) module. - -## Usage - -Available parameters. - -####`master` - -Configures a server to be an IPA master LDAP/Kerberos node. - -Defaults to 'false'. - -####`replica` - -Configures a server to be an IPA replica LDAP/Kerberos node. - -Defaults to 'false'. - -####`client` - -Configures a server to be an IPA client. - -Defaults to 'false'. - -####`cleanup` - -Removes IPA specific packages. - -Defaults to 'false'. - -####`domain` - -Defines the LDAP domain. - -Defaults to 'undef'. - -####`realm` - -Defines the Kerberos realm. - -Defaults to 'undef'. - -####`adminpw` - -Defines the IPA administrative user password. - -Defaults to 'undef'. - -####`dspw` - -Defines the IPA directory services password. - -Defaults to 'undef'. - -####`otp` - -Defines an IPA client one-time-password. - -Defaults to 'undef'. - -####`dns` - -Controls the option to configure a DNS zone with the IPA master setup. - -Defaults to 'false'. - -####`fixedprimary` - -Configure sssd to use a fixed server as the primary IPA server. - -Defaults to 'false'. - -####`forwarders` - -Defines an array of DNS forwarders to use when DNS is setup. An empty list will use the Root Nameservers. - -Defaults to '[]'. - -####`extca` - -Controls the option to configure an external CA. - -Defaults to 'false' - -####`extcertpath` - -Defines a file path to the external certificate file. Somewhere under /root is recommended. - -Defaults to 'undef' - -####`extcert` - -The X.509 certificate in base64 encoded format. - -Defaults to 'undef' - -####`extcapath` - -Defines a file path to the external CA certificate file. Somewhere under /root is recommended. - -Defaults ro 'undef' - -####`extcacert` - -The X.509 CA certificate in base64 encoded format. - -Defaults to 'undef' - -####`dirsrv_pkcs12` - -PKCS#12 file containing the Directory Server SSL Certificate, also corresponds to the Puppet fileserver path under fileserverconfig for $confdir/files/ipa - -Defaults to 'undef' - -####`http_pkcs12` - -The PKCS#12 file containing the Apache Server SSL Certificate, also corresponds to the Puppet fileserver path under fileserverconfig for $confdir/files/ipa - -Defaults to 'undef' - -####`dirsrv_pin` - -The password of the Directory Server PKCS#12 file. - -Defaults to 'undef' - -####`http_pin` - -The password of the Apache Server PKCS#12 file. - -Defaults to 'undef' - -####`subject` - -The certificate subject base. - -Defaults to 'undef' - -####`selfsign` - -Configure a self-signed CA instance for issuing server certificates instead of using dogtag for certificates. - -Defaults to 'false' - -####`loadbalance` - -Controls the option to include any additional hostnames to be used in a load balanced IPA client configuration. - -Defaults to 'false'. - -####`ipaservers` - -Defines an array of additional hostnames to be used in a load balanced IPA client configuration. - -Defaults to '[]' - -####`mkhomedir` - -Controls the option to create user home directories on first login. - -Defaults to 'false'. - -####`ntp` - -Controls the option to configure NTP on a client. - -Defaults to 'false'. - -####`kstart` - -Controls the installation of kstart. - -Defaults to 'true'. - -####`desc` - -Controls the description entry of an IPA client. - -Defaults to ''. - -####`locality` - -Controls the locality entry of an IPA client. - -Defaults to ''. - -####`location` - -Controls the location entry of an IPA client. - -Defaults to ''. - -####`sssdtools` - -Controls the installation of the SSSD tools package. - -Defaults to 'true'. - -####`sssdtoolspkg` - -SSSD tools package name. - -Defaults to 'sssd-tools' - -####`sssd` - -Controls the option to start the SSSD service if its not defined elsewhere. Note: Set to false if the SSSD service is defined in your site specific modules. - -Defaults to 'true'. - -####`sudo` - -Controls the option to configure sudo in LDAP. - -Defaults to 'false'. - -####`sudopw` - -Defines the sudo user bind password. - -Defaults to 'undef'. - -####`debiansudopkg` - -Controls the installation of the Debian sudo-ldap package. - -Defaults to 'true'. - -####`automount` - -Controls the option to configure automounter maps in LDAP. - -Defaults to 'false'. - -####`autofs` - -Controls the option to start the autofs service and install the autofs package. - -Defaults to 'false'. - -####`svrpkg` - -IPA server package name. - -Defaults to 'ipa-server'. - -####`clntpkg` - -IPA client package name. - -Defaults to 'ipa-client'. - -####`ldaputils` - -Controls the instalation of the LDAP utilities package. - -Defaults to 'true'. - -####`ldaputilspkg` - -LDAP utilities package name. - -Defaults to 'openldap-clients'. - -## Usage examples - -Here are a few simple usage examples. If you don't want to put your passwords in the clear, then use hiera/gpg. - -IPA master: - -```puppet - node 'ipamaster.domain.name' { - class { 'ipa': - master => true, # Only one master per Puppet master - domain => 'domain.name', - realm => 'DOMAIN.NAME', - adminpw => 'somepasswd', # Cleartext example - dspw => hiera('some_passwd') # Using hiera - } - } -``` - -IPA replica: - -```puppet - node 'ipareplica1.domain.name' { - class { 'ipa': - replica => true, # Multiple replicas can be setup. - domain => 'domain.name', - realm => 'DOMAIN.NAME', - adminpw => 'somepasswd', - dspw => 'somepasswd', - otp => 'onetimepasswd' - } - } -``` - -Another IPA replica: - -```puppet - node 'ipareplica2.domain.name' { - class { 'ipa': - replica => true, - domain => 'domain.name', - realm => 'DOMAIN.NAME', - adminpw => hiera('some_passwd'), - dspw => hiera('some_passwd'), - otp => hiera('one_time_passwd') - } - } -``` - -IPA client: - -```puppet - node 'ipaclient.domain.name' { - class { 'ipa': - client => true, - domain => 'domain.name', - realm => 'DOMAIN.NAME', - loadbalance => true, - ipaservers => ['ipaloadbalanceddnsname.domain.name','ipamaster.domain.name','ipareplica1.domain.name','ipareplica2.domain.name'], - desc => 'This is an IPA client', # This string will show up the the description attribute of the computer account. - otp => hiera('one_time_passwd') - } - } -``` - -Cleanup parameter: - -```puppet - node 'ipawhatever.domain.name' { - class { 'ipa': - cleanup => true # Removes IPA completely. Mutually exclusive from master, replica and client parameters. - } - } -``` - -## Limitations - -IPA master and replicas require a RedHat family OS. - -Client configuration does not work with Ubuntu 8.04 and Ubuntu 10.04 - -## License - - huit/puppet-ipa - Puppet module that can manage an IPA master, replicas and clients. - - Copyright (C) 2013 Harvard University Information Technology - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program. If not, see . - -## Support - -Please report issues [here](https://github.com/huit/puppet-ipa/issues). - -For more information see [https://github.com/huit/puppet-ipa.git](https://github.com/huit/puppet-ipa.git) diff --git a/ipa/Rakefile b/ipa/Rakefile deleted file mode 100644 index 8f7a2b50c..000000000 --- a/ipa/Rakefile +++ /dev/null @@ -1,11 +0,0 @@ -require 'rubygems' -require 'rake' -require 'rspec/core/rake_task' -require 'puppetlabs_spec_helper/rake_tasks' -require 'puppet-lint' - -PuppetLint.configuration.ignore_paths = ["pkg/**/*.pp", "tests/*.pp"] -PuppetLint.configuration.send('disable_80chars') -PuppetLint.configuration.send('disable_documentation') -PuppetLint.configuration.send('disable_class_inherits_from_params_class') -task :default => [:spec, :lint] diff --git a/ipa/lib/facter/ipa_adminhomedir.rb b/ipa/lib/facter/ipa_adminhomedir.rb deleted file mode 100644 index 61b86c288..000000000 --- a/ipa/lib/facter/ipa_adminhomedir.rb +++ /dev/null @@ -1,10 +0,0 @@ -Facter.add(:ipa_adminhomedir) do - setcode do - require 'etc' - begin - Etc.getpwnam('admin').dir - rescue ArgumentError => ae - '' - end - end -end diff --git a/ipa/lib/facter/ipa_adminuidnumber.rb b/ipa/lib/facter/ipa_adminuidnumber.rb deleted file mode 100644 index ca18f558b..000000000 --- a/ipa/lib/facter/ipa_adminuidnumber.rb +++ /dev/null @@ -1,10 +0,0 @@ -Facter.add(:ipa_adminuidnumber) do - setcode do - require 'etc' - begin - Etc.getpwnam('admin').uid - rescue ArgumentError => ae - '' - end - end -end diff --git a/ipa/lib/facter/ipa_replicascheme.rb b/ipa/lib/facter/ipa_replicascheme.rb deleted file mode 100644 index a5006c91d..000000000 --- a/ipa/lib/facter/ipa_replicascheme.rb +++ /dev/null @@ -1,16 +0,0 @@ -Facter.add(:ipa_replicascheme, :timeout => 10) do - setcode do - host = Facter.value(:hostname) - domain = Facter.value(:domain) - if File.exists?('/etc/ipa/primary') and File.exist?(File.expand_path('~admin')) and File.exists?(File.expand_path('~admin/admin.keytab')) - if host and domain - fqdn = [host, domain].join(".") - end - servers = Facter::Util::Resolution.exec("/sbin/runuser -l admin -c '/usr/sbin/ipa-replica-manage list' 2>/dev/null | /bin/egrep -v '#{fqdn}|winsync' | /bin/cut -d: -f1") - combinations = servers.scan(/[\w.-]+/).combination(2).to_a - combinations.collect { |combination| combination.join(',') }.join(':') - else - nil - end - end -end diff --git a/ipa/lib/puppet/parser/functions/ipa_string2hash.rb b/ipa/lib/puppet/parser/functions/ipa_string2hash.rb deleted file mode 100644 index b904a9696..000000000 --- a/ipa/lib/puppet/parser/functions/ipa_string2hash.rb +++ /dev/null @@ -1,17 +0,0 @@ -module Puppet::Parser::Functions - newfunction(:ipa_string2hash, :type => :rvalue, :doc => 'This function defines the IPA replica scheme by converting a comma delimited string separated by a colon into a hash. ex. "string1,string2:string3,string4"') do |arguments| - if arguments.size != 1 - raise(Puppet::ParseError, "suffix(): Wrong number of arguments " + "given (#{arguments.size} for 1)") - else - packed = arguments[0] - hashed = {} - unless packed.is_a?(String) - raise(Puppet::ParseError, 'ipa_string2hash(): Requires string to work with') - end - pairs = packed.split(':').map { |pair| pair.split(',') } - hashed = {} - pairs.each { |from, to| hashed["#{from}-#{to}"] = {'to' => to,'from' => from} } - end - return hashed - end -end diff --git a/ipa/manifests/adminconfig.pp b/ipa/manifests/adminconfig.pp deleted file mode 100644 index 4719071fb..000000000 --- a/ipa/manifests/adminconfig.pp +++ /dev/null @@ -1,64 +0,0 @@ -define ipa::adminconfig ( - $host = $name, - $idstart = {}, - $realm = {}, -) { - - $adminuidnumber = is_numeric($::ipa_adminuidnumber) ? { - false => $idstart, - default => $::ipa_adminuidnumber - } - - $adminhomedir = $::ipa_adminhomedir ? { - '' => '/home/admin', - default => $::ipa_adminhomedir - } - - k5login { "${adminhomedir}/.k5login": - principals => $ipa::master::principals, - notify => File["${adminhomedir}/.k5login"], - require => File[$adminhomedir] - } - - $kadminlocalcmd = shellquote('/usr/sbin/kadmin.local','-q',"ktadd -norandkey -k ${adminhomedir}/admin.keytab admin") - $chownkeytabcmd = shellquote('/usr/bin/chown',"${adminuidnumber}:${adminuidnumber}","${adminhomedir}/admin.keytab") - $k5startcmd = shellquote('/sbin/runuser','-l','admin','-c',"/usr/bin/k5start -f ${adminhomedir}/admin.keytab -U") - - exec { 'admin_keytab': - command => "${kadminlocalcmd} && ${chownkeytabcmd} && ${k5startcmd} > /dev/null 2>&1", - cwd => $adminhomedir, - unless => shellquote('/usr/bin/kvno','-k',"${adminhomedir}/admin.keytab","admin@${realm}"), - notify => File["${adminhomedir}/admin.keytab"], - require => Cron['k5start_admin'] - } - - cron { 'k5start_admin': - command => "/usr/bin/k5start -f ${adminhomedir}/admin.keytab -U > /dev/null 2>&1", - user => 'admin', - minute => '*/1', - require => [Package['kstart'], K5login["${adminhomedir}/.k5login"], File[$adminhomedir]] - } - - file { $adminhomedir: - ensure => directory, - mode => '0700', - owner => $adminuidnumber, - group => $adminuidnumber, - recurse => true, - notify => Exec['admin_keytab'], - require => Exec["serverinstall-${host}"] - } - - file { "${adminhomedir}/.k5login": - owner => $adminuidnumber, - group => $adminuidnumber, - require => File[$adminhomedir] - } - - file { "${adminhomedir}/admin.keytab": - owner => $adminuidnumber, - group => $adminuidnumber, - mode => '0600', - require => File[$adminhomedir] - } -} diff --git a/ipa/manifests/cleanup.pp b/ipa/manifests/cleanup.pp deleted file mode 100644 index 6f5847cc8..000000000 --- a/ipa/manifests/cleanup.pp +++ /dev/null @@ -1,44 +0,0 @@ -# Definition: ipa::cleanup -# -# Cleans up an IPA installation -define ipa::cleanup ( - $svrpkg = {}, - $clntpkg = {} -) { - - $pkgrmcmd = $::osfamily ? { - RedHat => '/usr/bin/yum -y remove', - Debian => '/usr/bin/aptitude -y purge' - } - - $pkgcmd = regsubst($pkgrmcmd,'\s.*$','') - - Cron['k5start_admin'] -> Cron['k5start_root'] -> Exec["cleanup-${name}"] - - exec { "cleanup-${name}": - command => "/bin/bash -c \"if [ -x /usr/sbin/ipactl ]; then /usr/sbin/ipactl stop ; fi ;\ - if [ -n \"${::ipa_adminhomedir}\" ] && [ -d ${::ipa_adminhomedir} ]; then /bin/rm -rf ${::ipa_adminhomedir} ; fi ;\ - if [ -x /usr/sbin/ipa-client-install ]; then /bin/echo | /usr/sbin/ipa-client-install --uninstall --unattended ; fi ;\ - if [ -x /usr/sbin/ipa-server-install ]; then /usr/sbin/ipa-server-install --uninstall --unattended ; fi ;\ - if [ -d /var/lib/pki-ca ]; then /usr/bin/pkiremove -pki_instance_root=/var/lib -pki_instance_name=pki-ca -force ; fi ;\ - if [ -x ${pkgcmd} ]; then ${pkgrmcmd} ${svrpkg} ${clntpkg} krb5-server 389-ds-base 389-ds-base-libs pki-ca pki-util pki-ca certmonger pki-native-tools pki-symkey pki-setup ipa-pki-common-theme pki-selinux ipa-pki-ca-theme ipa-python ; fi ;\ - if [ -e /etc/openldap/ldap.conf.ipabkp ]; then /bin/cp -f /etc/openldap/ldap.conf.ipabkp /etc/openldap/ldap.conf ; fi ;\ - if [ -e /etc/krb5.conf.ipabkp ]; then /bin/cp -f /etc/krb5.conf.ipabkp /etc/krb5.conf ; fi ;\ - if [ -e /etc/krb5.keytab ]; then /bin/mv -f /etc/krb5.keytab /etc/krb5.keytab.puppet-ipa.cleanup ; fi ;\ - if [ -e /root/ipa.csr ]; then /bin/mv -f /root/ipa.csr /root/ipa.csr.$(/bin/date +%s) ; fi ;\ - if [ -d /var/lib/certmonger ]; then find /var/lib/certmonger -type f -exec /bin/rm -f '{}' \; ; fi ;\ - if [ -d /var/lib/ipa ]; then /usr/bin/find /var/lib/ipa -type f -exec /bin/rm -f '{}' \; ; fi ;\ - if [ -d /var/lib/ipa-client ]; then /usr/bin/find /var/lib/ipa-client -type f -exec /bin/rm -f '{}' \; ; fi ;\ - if [ -d /etc/ipa ]; then /usr/bin/find /etc/ipa -type f -exec /bin/rm -f '{}' \; ; fi\"", - timeout => '0', - logoutput => true - } - - Cron <|title == 'k5start_root'|> { - ensure => 'absent', - require => undef - } - - notify { 'Running IPA install cleanup, please wait.': } -> - cron { 'k5start_admin': ensure => 'absent';} -} diff --git a/ipa/manifests/client.pp b/ipa/manifests/client.pp deleted file mode 100644 index db943a7e0..000000000 --- a/ipa/manifests/client.pp +++ /dev/null @@ -1,155 +0,0 @@ -# Class: ipa::client -# -# This class configures an IPA client -# -# Parameters: -# -# Actions: -# -# Requires: Exported resources, puppetlabs/puppetlabs-firewall, puppetlabs/stdlib -# -# Sample Usage: -# -class ipa::client ( - $clntpkg = {}, - $ldaputils = {}, - $ldaputilspkg = {}, - $sssdtools = {}, - $sssdtoolspkg = {}, - $sssd = {}, - $kstart = true, - $client = {}, - $domain = {}, - $realm = {}, - $sudo = {}, - $debiansudopkg = {}, - $automount = {}, - $autofs = {}, - $otp = {}, - $ipaservers = [], - $loadbalance = {}, - $mkhomedir = false, - $ntp = false, - $fixedprimary = false, - $desc = {}, - $locality = {}, - $location = {} -) { - - if $ipa::client::kstart { - Package['kstart'] -> Ipa::Clientinstall<||> - realize Package['kstart'] - } - - Ipa::Clientinstall <<| |>> { - name => $::fqdn, - otp => $ipa::client::otp, - domain => $ipa::client::domain, - mkhomedir => $ipa::client::mkhomedir, - ntp => $ipa::client::ntp, - fixedprimary => $ipa::client::fixedprimary, - require => Package[$ipa::client::clntpkg], - } - - if $ipa::client::sudo { - Ipa::Configsudo <<| |>> { - name => $::fqdn, - os => "${::osfamily}${::lsbmajdistrelease}", - require => Ipa::Clientinstall[$::fqdn] - } - } - - if $ipa::client::automount { - if $ipa::client::autofs { - realize Service['autofs'] - realize Package['autofs'] - } - - Ipa::Configautomount <<| |>> { - name => $::fqdn, - os => $::osfamily, - notify => Service['autofs'], - require => Ipa::Clientinstall[$::fqdn] - } - } - - if defined(Package[$ipa::client::clntpkg]) { - realize Package[$ipa::client::clntpkg] - } - - if $ipa::client::ldaputils { - if defined(Package[$ipa::client::ldaputilspkg]) { - realize Package[$ipa::client::ldaputilspkg] - } - } - - if $ipa::client::sssdtools { - if defined(Package[$ipa::client::sssdtoolspkg]) { - realize Package[$ipa::client::sssdtoolspkg] - } - } - - if $ipa::client::sssd { - Ipa::Clientinstall<||> -> Service['sssd'] - realize Package['sssd-common'] - realize Service['sssd'] - } - - if $::osfamily == 'Debian' { - file { '/etc/pki': - ensure => 'directory', - mode => '0755', - owner => 'root', - group => 'root', - require => Package[$ipa::client::clntpkg] - } - - file {'/etc/pki/nssdb': - ensure => 'directory', - mode => '0755', - owner => 'root', - group => 'root', - require => File['/etc/pki'] - } - - File['/etc/pki/nssdb'] -> Ipa::Clientinstall <<| |>> - - if $ipa::client::sudo and $ipa::client::debiansudopkg { - @package { 'sudo-ldap': - ensure => installed - } - realize Package['sudo-ldap'] - } - - if $ipa::client::mkhomedir == true { - augeas { - 'mkhomedir_pam' : - context => '/files/etc/pam.d/common-session', - changes => ['ins 1000000 after *[last()]', - 'set 1000000/type session', - 'set 1000000/control required', - 'set 1000000/module pam_mkhomedir.so', - 'set 1000000/argument umask=0022'], - onlyif => 'match *[type="session"][module="pam_mkhomedir.so"][argument="umask=0022"] size == 0' - } - } - } - - @@ipa::hostadd { $::fqdn: - otp => $ipa::client::otp, - desc => $ipa::client::desc, - clientos => $::lsbdistdescription, - clientpf => $::manufacturer, - locality => $ipa::client::locality, - location => $ipa::client::location - } - - if $ipa::client::loadbalance { - ipa::loadbalanceconf { "client-${::fqdn}": - domain => $ipa::client::domain, - ipaservers => $ipa::client::ipaservers, - mkhomedir => $ipa::client::mkhomedir, - require => Ipa::Clientinstall[$::fqdn] - } - } -} diff --git a/ipa/manifests/clientinstall.pp b/ipa/manifests/clientinstall.pp deleted file mode 100644 index 84c33ac8b..000000000 --- a/ipa/manifests/clientinstall.pp +++ /dev/null @@ -1,49 +0,0 @@ -# Definition: ipa::clientinstall -# -# Installs an IPA client -define ipa::clientinstall ( - $host = $name, - $masterfqdn = {}, - $domain = {}, - $realm = {}, - $adminpw = {}, - $otp = {}, - $mkhomedir = {}, - $ntp = {}, - $fixedprimary = false -) { - - Exec["client-install-${host}"] ~> Ipa::Flushcache["client-${host}"] - - $mkhomediropt = $mkhomedir ? { - true => '--mkhomedir', - default => '' - } - - $ntpopt = $ntp ? { - true => '', - default => '--no-ntp' - } - - $fixedprimaryopt = $fixedprimary ? { - true => '--fixed-primary', - default => '' - } - - $clientinstallcmd = shellquote('/usr/sbin/ipa-client-install',"--server=${masterfqdn}","--hostname=${host}","--domain=${domain}","--realm=${realm}","--password=${otp}",$mkhomediropt,$ntpopt,$fixedprimaryopt,'--unattended') - $dc = prefix([regsubst($domain,'(\.)',',dc=','G')],'dc=') - $searchostldapcmd = shellquote('/usr/bin/k5start','-u',"host/${host}",'-f','/etc/krb5.keytab','--','/usr/bin/ldapsearch','-Y','GSSAPI','-H',"ldap://${masterfqdn}",'-b',$dc,"fqdn=${host}") - - exec { "client-install-${host}": - command => "/bin/echo | ${clientinstallcmd}", - unless => "${searchostldapcmd} | /bin/grep ^krbLastPwdChange", - timeout => '0', - tries => '60', - try_sleep => '90', - returns => ['0','1'], - logoutput => 'on_failure' - } - - ipa::flushcache { "client-${host}": - } -} diff --git a/ipa/manifests/configautomount.pp b/ipa/manifests/configautomount.pp deleted file mode 100644 index 04aefc782..000000000 --- a/ipa/manifests/configautomount.pp +++ /dev/null @@ -1,47 +0,0 @@ -define ipa::configautomount ( - $host = $name, - $os = {}, - $domain = {}, - $realm = {}, - $masterfqdn = {} -) { - - Augeas["nsswitch-automount-${host}"] -> Augeas["sysconfig-autofs-${name}"] -> File["autofs_ldap_auth-${host}"] - - $dc = prefix([regsubst($domain,'(\.)',',dc=','G')],'dc=') - - $autofspath = $os ? { - Debian => '/etc/default/autofs', - default => '/etc/sysconfig/autofs' - } - - augeas { "nsswitch-automount-${host}": - context => '/files/etc/nsswitch.conf', - changes => [ - 'set database[. = "automount"] automount', - 'set database[. = "automount"]/service[1] files', - 'set database[. = "automount"]/service[2] ldap', - ] - } - - augeas { "sysconfig-autofs-${name}": - context => $autofspath, - changes => [ - 'set MAP_OBJECT_CLASS automountMap', - 'set ENTRY_OBJECT_CLASS automount', - 'set MAP_ATTRIBUTE automountMapName', - 'set ENTRY_ATTRIBUTE automountKey', - 'set VALUE_ATTRIBUTE automountInformation', - "set LDAP_URI ldap://${masterfqdn}", - "set SEARCH_BASE cn=default,cn=automount,${dc}" - ] - } - - file { "autofs_ldap_auth-${host}": - path => '/etc/autofs_ldap_auth.conf', - owner => 'root', - group => 'root', - mode => '0600', - content => template('ipa/autofs_ldap_auth.conf.erb') - } -} diff --git a/ipa/manifests/configsudo.pp b/ipa/manifests/configsudo.pp deleted file mode 100644 index 8ad861eea..000000000 --- a/ipa/manifests/configsudo.pp +++ /dev/null @@ -1,62 +0,0 @@ -# Definition: ipa::configsudo -# -# Configures sudoers in LDAP -define ipa::configsudo ( - $host = $name, - $os = {}, - $sudopw = {}, - $adminpw = {}, - $domain = {}, - $masterfqdn = {} -) { - - Augeas["nsswitch-sudoers-${host}"] -> Exec["set-sudopw-${host}"] - - $dc = prefix([regsubst($domain,'(\.)',',dc=','G')],'dc=') - - augeas { "nsswitch-sudoers-${host}": - context => '/files/etc/nsswitch.conf', - changes => [ - 'set database[. = "sudoers"] sudoers', - 'set database[. = "sudoers"]/service[1] files', - 'set database[. = "sudoers"]/service[2] ldap' - ] - } - - if $os == 'RedHat5' { - augeas { "sudo-ldap-rhel5-${host}": - context => '/files/etc/ldap.conf', - changes => [ - "set binddn uid=sudo,cn=sysaccounts,cn=etc,${dc}", - "set bindpw ${sudopw}", - 'set ssl start_tls', - 'set tls_cacertfile /etc/ipa/ca.crt', - 'set tls_checkpeer yes', - 'set bind_timelimit 5', - 'set timelimit 15', - "set sudoers_base ou=sudoers,${dc}" - ] - } - } else { - file { "sudo-ldap-${host}": - path => '/etc/sudo-ldap.conf', - owner => 'root', - group => 'root', - mode => '0640', - content => template('ipa/sudo-ldap.conf.erb') - } - } - - exec { "set-sudopw-${host}": - command => "/bin/bash -c \"LDAPTLS_REQCERT=never /usr/bin/ldappasswd -x -H ldaps://${masterfqdn} -D uid=admin,cn=users,cn=accounts,${dc} -w ${adminpw} -s ${sudopw} uid=sudo,cn=sysaccounts,cn=etc,${dc}\"", - unless => "/bin/bash -c \"LDAPTLS_REQCERT=never /usr/bin/ldapsearch -x -H ldaps://${masterfqdn} -D uid=sudo,cn=sysaccounts,cn=etc,${dc} -w ${sudopw} -b cn=sysaccounts,cn=etc,${dc} uid=sudo\"", - onlyif => '/usr/sbin/ipactl status >/dev/null 2>&1', - logoutput => 'on_failure' - } - - exec { "setupnisdomain-${host}": - command => "/bin/nisdomainname ${domain}", - unless => "/usr/bin/test $(/bin/nisdomainname) = ${domain}", - require => Exec["set-sudopw-${host}"] - } -} diff --git a/ipa/manifests/createreplicas.pp b/ipa/manifests/createreplicas.pp deleted file mode 100644 index ca04dce26..000000000 --- a/ipa/manifests/createreplicas.pp +++ /dev/null @@ -1,6 +0,0 @@ -define ipa::createreplicas { - - $replicas = ipa_string2hash($::ipa_replicascheme) - create_resources('ipa::replicaagreement',$replicas) - -} diff --git a/ipa/manifests/flushcache.pp b/ipa/manifests/flushcache.pp deleted file mode 100644 index a3fd0b96c..000000000 --- a/ipa/manifests/flushcache.pp +++ /dev/null @@ -1,16 +0,0 @@ -define ipa::flushcache ( - $host = $name -) { - - $flushcmd = $::osfamily ? { - 'RedHat' => shellquote('if [ -x /usr/sbin/sss_cache ]; then /usr/sbin/sss_cache -UGNA >/dev/null 2>&1 ; else /usr/bin/find /var/lib/sss/db -type f -exec rm -f "{}" \; ; fi'), - 'Debian' => shellquote('if [ -x /usr/sbin/nscd ]; then /usr/sbin/nscd -i passwd -i group -i netgroup -i automount >/dev/null 2>&1 ; elif [ -x /usr/sbin/sss_cache ]; then /usr/sbin/sss_cache -UGNA >/dev/null 2>&1 ; else /usr/bin/find /var/lib/sss/db -type f -exec rm -f "{}" \; ; fi') - } - - exec { "flushcache-${host}": - command => "/bin/bash -c ${flushcmd}", - returns => ['0','1','2'], - notify => Service['sssd'], - refreshonly => true - } -} diff --git a/ipa/manifests/hostadd.pp b/ipa/manifests/hostadd.pp deleted file mode 100644 index 481a6ad1c..000000000 --- a/ipa/manifests/hostadd.pp +++ /dev/null @@ -1,21 +0,0 @@ -define ipa::hostadd ( - $host = $name, - $otp = {}, - $desc = {}, - $clientos = {}, - $clientpf = {}, - $locality = {}, - $location = {}, -) { - - $timestamp = strftime('%a %b %d %Y %r') - $descinfo = rstrip(join(['Added by HUIT IPA Puppet module on',$timestamp,$desc], ' ')) - - exec { "hostadd-${host}": - command => "/sbin/runuser -l admin -c \'/usr/bin/ipa host-add ${host} --force --locality=\"${locality}\" --location=\"${location}\" --desc=\"${descinfo}\" --platform=\"${clientpf}\" --os=\"${clientos}\" --password=${otp}\'", - unless => "/sbin/runuser -l admin -c \'/usr/bin/ipa host-show ${host} >/dev/null 2>&1\'", - tries => '60', - try_sleep => '60', - require => Ipa::Serverinstall[$::fqdn], - } -} diff --git a/ipa/manifests/hostdelete.pp b/ipa/manifests/hostdelete.pp deleted file mode 100644 index e0cfa40c9..000000000 --- a/ipa/manifests/hostdelete.pp +++ /dev/null @@ -1,10 +0,0 @@ -define ipa::hostdelete ( - $host = $name -) { - - exec { "hostdelete-${host}": - command => "/sbin/runuser -l admin -c \'/usr/bin/ipa host-del ${host}\'", - refreshonly => true, - onlyif => "/sbin/runuser -l admin -c \'/usr/bin/ipa host-show ${host} >/dev/null 2>&1\'" - } -} diff --git a/ipa/manifests/init.pp b/ipa/manifests/init.pp deleted file mode 100644 index 51b7040a6..000000000 --- a/ipa/manifests/init.pp +++ /dev/null @@ -1,350 +0,0 @@ -# == Class: ipa -# -# Manages IPA masters, replicas and clients. -# -# === Parameters -# -# $master = false - Configures a server to be an IPA master LDAP/Kerberos node. -# $replica = false - Configures a server to be an IPA replica LDAP/Kerberos node. -# $client = false - Configures a server to be an IPA client. -# $cleanup = false - Removes IPA specific packages. -# $domain = undef - Defines the LDAP domain. -# $realm = undef - Defines the Kerberos realm. -# $adminpw = undef - Defines the IPA administrative user password. -# $dspw = undef - Defines the IPA directory services password. -# $otp = undef - Defines an IPA client one-time-password. -# $dns = false - Controls the option to configure a DNS zone with the IPA master setup. -# $fixedprimary = false - Configure sssd to use a fixed server as the primary IPA server. -# $forwarders = [] - Defines an array of DNS forwarders to use when DNS is setup. An empty list will use the Root Nameservers. -# $extca = false - Controls the option to configure an external CA. -# $extcertpath = undef - Defines a file path to the external certificate file. Somewhere under /root is recommended. -# $extcert = undef - The X.509 certificate in base64 encoded format. -# $extcacertpath = undef - Defines a file path to the external CA certificate file. Somewhere under /root is recommended. -# $extcacert = undef - The X.509 CA certificate in base64 encoded format. -# $dirsrv_pkcs12 = undef - PKCS#12 file containing the Directory Server SSL Certificate, also corresponds to the Puppet fileserver path under fileserverconfig for $confdir/files/ipa -# $http_pkcs12 = undef - The PKCS#12 file containing the Apache Server SSL Certificate, also corresponds to the Puppet fileserver path under fileserverconfig for $confdir/files/ipa -# $dirsrv_pin = undef - The password of the Directory Server PKCS#12 file. -# $http_pin = undef - The password of the Apache Server PKCS#12 file. -# $subject = undef - The certificate subject base. -# $selfsign = false - Configure a self-signed CA instance for issuing server certificates instead of using dogtag for certificates. -# $loadbalance = false - Controls the option to include any additional hostnames to be used in a load balanced IPA client configuration. -# $ipaservers = [] - Defines an array of additional hostnames to be used in a load balanced IPA client configuration. -# $mkhomedir = false - Controls the option to create user home directories on first login. -# $ntp = false - Controls the option to configure NTP on a client. -# $kstart = true - Controls the installation of kstart. -# $desc = '' - Controls the description entry of an IPA client. -# $locality = '' - Controls the locality entry of an IPA client. -# $location = '' - Controls the location entry of an IPA client. -# $sssdtools = true - Controls the installation of the SSSD tools package. -# $sssdtoolspkg = 'sssd-tools' - SSSD tools package. -# $sssd = true - Controls the option to start the SSSD service. -# $sudo = false - Controls the option to configure sudo in LDAP. -# $sudopw = undef - Defines the sudo user bind password. -# $debiansudopkg = true - Controls the installation of the Debian sudo-ldap package. -# $automount = false - Controls the option to configure automounter maps in LDAP. -# $autofs = false - Controls the option to start the autofs service and install the autofs package. -# $svrpkg = 'ipa-server' - IPA server package. -# $clntpkg = 'ipa-client' - IPA client package. -# $ldaputils = true - Controls the instalation of the LDAP utilities package. -# $ldaputilspkg = 'openldap-clients' - LDAP utilities package. -# -# === Variables -# -# -# === Examples -# -# -# === Authors -# -# -# === Copyright -# -# -class ipa ( - $master = false, - $replica = false, - $client = false, - $cleanup = false, - $domain = undef, - $realm = undef, - $adminpw = undef, - $dspw = undef, - $otp = undef, - $dns = false, - $fixedprimary = false, - $forwarders = [], - $extca = false, - $extcertpath = undef, - $extcert = undef, - $extcacertpath = undef, - $extcacert = undef, - $dirsrv_pkcs12 = undef, - $http_pkcs12 = undef, - $dirsrv_pin = undef, - $http_pin = undef, - $subject = undef, - $selfsign = false, - $loadbalance = false, - $ipaservers = [], - $mkhomedir = false, - $ntp = false, - $kstart = true, - $desc = '', - $locality = '', - $location = '', - $sssdtools = true, - $sssdtoolspkg = 'sssd-tools', - $sssd = true, - $sudo = false, - $sudopw = undef, - $debiansudopkg = true, - $automount = false, - $autofs = false, - $svrpkg = 'ipa-server', - $clntpkg = $::osfamily ? { - Debian => 'freeipa-client', - default => 'ipa-client', - }, - $ldaputils = true, - $ldaputilspkg = $::osfamily ? { - Debian => 'ldap-utils', - default => 'openldap-clients', - }, - $idstart = false -) { - - @package { $ipa::svrpkg: - ensure => installed - } - - @package { $ipa::clntpkg: - ensure => installed - } - - if $ipa::ldaputils { - @package { $ipa::ldaputilspkg: - ensure => installed - } - } - - if $ipa::sssdtools { - @package { $ipa::sssdtoolspkg: - ensure => installed - } - } - - if $ipa::kstart { - @package { 'kstart': - ensure => installed - } - } - - @service { 'ipa': - ensure => 'running', - enable => true, - require => Package[$ipa::svrpkg] - } - - if $ipa::sssd { - @package { 'sssd-common': - ensure => installed - } - - @service { 'sssd': - ensure => 'running', - enable => true, - require => Package['sssd-common'] - } - } - - if $ipa::dns { - @package { 'bind-dyndb-ldap': - ensure => installed - } - } - - if $ipa::mkhomedir and $::osfamily == 'RedHat' and $::lsbmajdistrelease == '6' { - service { 'oddjobd': - ensure => 'running', - enable => true - } - } - - if $ipa::autofs { - @package { 'autofs': - ensure => installed - } - - @service { 'autofs': - ensure => 'running', - enable => true - } - } - - @cron { 'k5start_root': - command => '/usr/bin/k5start -f /etc/krb5.keytab -U -o root -k /tmp/krb5cc_0 > /dev/null 2>&1', - user => 'root', - minute => '*/1', - require => Package['kstart'] - } - - if $ipa::master and $ipa::replica { - fail('Conflicting options selected. Cannot configure both master and replica at once.') - } - - if ! $ipa::cleanup { - if $ipa::master or $ipa::replica { - validate_re($ipa::adminpw,'^.........*$','Parameter "adminpw" must be at least 8 characters long') - validate_re($ipa::dspw,'^.........*$','Parameter "dspw" must be at least 8 characters long') - } - if $ipa::master and $ipa::idstart { - validate_re($ipa::idstart,'^......*$', 'Parameter "idstart" must be an integer greater than 10000 ') - validate_re($ipa::idstart,'^\d+$', 'Parameter "idstart" must be an integer ') - } - - if ! $ipa::domain { - fail('Required parameter "domain" missing') - } - - if ! $ipa::realm { - fail('Required parameter "realm" missing') - } - - if ! is_domain_name($ipa::domain) { - fail('Parameter "domain" is not a valid domain name') - } - - if ! is_domain_name($ipa::realm) { - fail('Parameter "realm" is not a valid domain name') - } - } - - if $ipa::cleanup { - if $ipa::master or $ipa::replica or $ipa::client { - fail('Conflicting options selected. Cannot cleanup during an installation.') - } else { - ipa::cleanup { $::fqdn: - svrpkg => $ipa::svrpkg, - clntpkg => $ipa::clntpkg - } - } - } - - if $ipa::master { - class { 'ipa::master': - svrpkg => $ipa::svrpkg, - dns => $ipa::dns, - forwarders => $ipa::forwarders, - domain => downcase($ipa::domain), - realm => upcase($ipa::realm), - adminpw => $ipa::adminpw, - dspw => $ipa::dspw, - loadbalance => $ipa::loadbalance, - ipaservers => $ipa::ipaservers, - sudo => $ipa::sudo, - sudopw => $ipa::sudopw, - automount => $ipa::automount, - autofs => $ipa::autofs, - kstart => $ipa::kstart, - sssd => $ipa::sssd, - ntp => $ipa::ntp, - extca => $ipa::extca, - extcertpath => $ipa::extcertpath, - extcert => $ipa::extcert, - extcacertpath => $ipa::extcacertpath, - extcacert => $ipa::extcacert, - dirsrv_pkcs12 => $ipa::dirsrv_pkcs12, - http_pkcs12 => $ipa::http_pkcs12, - dirsrv_pin => $ipa::dirsrv_pin, - http_pin => $ipa::http_pin, - subject => $ipa::subject, - selfsign => $ipa::selfsign, - idstart => $ipa::idstart - } - - if ! $ipa::adminpw { - fail('Required parameter "adminpw" missing') - } - - if ! $ipa::dspw { - fail('Required parameter "dspw" missing') - } - } - - if $ipa::replica { - class { 'ipa::replica': - svrpkg => $ipa::svrpkg, - domain => downcase($ipa::domain), - adminpw => $ipa::adminpw, - dspw => $ipa::dspw, - kstart => $ipa::kstart, - sssd => $ipa::sssd - } - - class { 'ipa::client': - clntpkg => $ipa::clntpkg, - ldaputils => $ipa::ldaputils, - ldaputilspkg => $ipa::ldaputilspkg, - sssdtools => $ipa::sssdtools, - sssdtoolspkg => $ipa::sssdtoolspkg, - sssd => $ipa::sssd, - kstart => $ipa::kstart, - loadbalance => $ipa::loadbalance, - ipaservers => $ipa::ipaservers, - mkhomedir => $ipa::mkhomedir, - domain => downcase($ipa::domain), - realm => upcase($ipa::realm), - otp => $ipa::otp, - sudo => $ipa::sudo, - automount => $ipa::automount, - autofs => $ipa::autofs, - ntp => $ipa::ntp, - fixedprimary => $ipa::fixedprimary, - desc => $ipa::desc, - locality => $ipa::locality, - location => $ipa::location - } - - if ! $ipa::adminpw { - fail('Required parameter "adminpw" missing') - } - - if ! $ipa::dspw { - fail('Required parameter "dspw" missing') - } - - if ! $ipa::otp { - fail('Required parameter "otp" missing') - } - } - - if $ipa::client { - class { 'ipa::client': - clntpkg => $ipa::clntpkg, - ldaputils => $ipa::ldaputils, - ldaputilspkg => $ipa::ldaputilspkg, - sssdtools => $ipa::sssdtools, - sssdtoolspkg => $ipa::sssdtoolspkg, - sssd => $ipa::sssd, - domain => downcase($ipa::domain), - realm => upcase($ipa::realm), - otp => $ipa::otp, - sudo => $ipa::sudo, - debiansudopkg => $ipa::debiansudopkg, - automount => $ipa::automount, - autofs => $ipa::autofs, - mkhomedir => $ipa::mkhomedir, - loadbalance => $ipa::loadbalance, - ipaservers => $ipa::ipaservers, - ntp => $ipa::ntp, - fixedprimary => $ipa::fixedprimary, - desc => $ipa::desc, - locality => $ipa::locality, - location => $ipa::location - } - - if ! $ipa::otp { - fail('Required parameter "otp" missing') - } - } -} diff --git a/ipa/manifests/loadbalanceconf.pp b/ipa/manifests/loadbalanceconf.pp deleted file mode 100644 index 71f3eb959..000000000 --- a/ipa/manifests/loadbalanceconf.pp +++ /dev/null @@ -1,34 +0,0 @@ -# Definition: ipa::loadbalanceconf -# -# Configures IPA load balancing -define ipa::loadbalanceconf ( - $host = $name, - $domain = {}, - $ipaservers = [], - $mkhomedir = {} -) { - - $dc = prefix([regsubst($domain,'(\.)',',dc=','G')],'dc=') - - $servers = chop(inline_template('<% @ipaservers.each do |ipaserver| -%><%= ipaserver %>,<% end -%>')) - - case $::osfamily { - 'Debian': { - notify { 'Unable to configure load balanced IPA directory services for Debian.': } - } - default: { - $mkhomediropt = $mkhomedir ? { - true => '--enablemkhomedir', - default => '' - } - - exec { "loadbalanceconf-authconfig-${host}": - command => "/usr/sbin/authconfig --nostart --enablesssd --enableldap --ldapserver=${servers} --ldapbasedn=${dc} --krb5kdc=${servers} --krb5adminserver=${servers} ${mkhomediropt} --update", - logoutput => 'on_failure' - } ~> Ipa::Flushcache["loadbalanceconf-flushcache-${host}"] - } - } - - ipa::flushcache { "loadbalanceconf-flushcache-${host}": - } -} diff --git a/ipa/manifests/master.pp b/ipa/manifests/master.pp deleted file mode 100644 index 5746593aa..000000000 --- a/ipa/manifests/master.pp +++ /dev/null @@ -1,227 +0,0 @@ -# Class: ipa::master -# -# This class configures an IPA master -# -# Parameters: -# -# Actions: -# -# Requires: Exported resources, puppetlabs/puppetlabs-firewall, puppetlabs/stdlib -# -# Sample Usage: -# -class ipa::master ( - $svrpkg = {}, - $dns = {}, - $forwarders = [], - $realm = {}, - $domain = {}, - $ipaservers = [], - $loadbalance = {}, - $adminpw = {}, - $dspw = {}, - $sudo = {}, - $sudopw = {}, - $automount = {}, - $autofs = {}, - $kstart = {}, - $sssd = {}, - $ntp = {}, - $extca = {}, - $extcertpath = {}, - $extcert = {}, - $extcacertpath = {}, - $extcacert = {}, - $dirsrv_pkcs12 = {}, - $http_pkcs12 = {}, - $dirsrv_pin = {}, - $http_pin = {}, - $subject = {}, - $selfsign = {}, - $idstart = {} -) { - - Ipa::Serverinstall[$::fqdn] -> File['/etc/ipa/primary'] -> Ipa::Hostadd <<| |>> -> Ipa::Replicareplicationfirewall <<| tag == "ipa-replica-replication-firewall-${ipa::master::domain}" |>> -> Ipa::Replicaprepare <<| tag == "ipa-replica-prepare-${ipa::master::domain}" |>> -> Ipa::Createreplicas[$::fqdn] - - Ipa::Replicareplicationfirewall <<| tag == "ipa-replica-replication-firewall-${ipa::master::domain}" |>> - Ipa::Replicaprepare <<| tag == "ipa-replica-prepare-${ipa::master::domain}" |>> - Ipa::Hostadd <<| |>> - - file { '/etc/ipa/primary': - ensure => 'file', - content => 'Added by HUIT IPA Puppet module: designates primary master - do not remove.' - } - - if $ipa::master::sudo { - Ipa::Configsudo <<| |>> { - name => $::fqdn, - os => "${::osfamily}${::lsbmajdistrelease}", - require => Ipa::Serverinstall[$::fqdn] - } - } - - if $ipa::master::automount { - if $ipa::master::autofs { - realize Service['autofs'] - realize Package['autofs'] - } - - Ipa::Configautomount <<| |>> { - name => $::fqdn, - os => $::osfamily, - notify => Service['autofs'], - require => Ipa::Serverinstall[$::fqdn] - } - } - - $principals = suffix(prefix([$::fqdn], 'host/'), "@${ipa::master::realm}") - - if $::osfamily != 'RedHat' { - fail("Cannot configure an IPA master server on ${::operatingsystem} operating systems. Must be a RedHat-like operating system.") - } - - realize Package[$ipa::master::svrpkg] - - if $ipa::master::sssd { - realize Package['sssd-common'] - realize Service['sssd'] - } - - if $ipa::master::kstart { - realize Package['kstart'] - } - - if $ipa::master::dns { - if size($ipa::master::forwarders) > 0 { - $forwarderopts = join(prefix($ipa::master::forwarders, '--forwarder '), ' ') - } - else { - $forwarderopts = '--no-forwarders' - } - $dnsopt = '--setup-dns' - realize Package['bind-dyndb-ldap'] - } - else { - $dnsopt = '' - $forwarderopts = '' - } - - $ntpopt = $ipa::master::ntp ? { - false => '--no-ntp', - default => '' - } - - $extcaopt = $extca ? { - true => '--external-ca', - default => '' - } - - $random_idstart = fqdn_rand('10737') + 10000 - - $generated_idstart = $idstart ? { - false => $random_idstart, - default => $idstart, - } - - ipa::serverinstall { $::fqdn: - realm => $ipa::master::realm, - domain => $ipa::master::domain, - adminpw => $ipa::master::adminpw, - dspw => $ipa::master::dspw, - dnsopt => $ipa::master::dnsopt, - forwarderopts => $ipa::master::forwarderopts, - ntpopt => $ipa::master::ntpopt, - extcaopt => $ipa::master::extcaopt, - idstart => $ipa::master::generated_idstart, - require => Package[$ipa::master::svrpkg] - } - - if $extca { - class { 'ipa::master_extca': - extcertpath => $ipa::master::extcertpath, - extcert => $ipa::master::extcert, - extcacertpath => $ipa::master::extcacertpath, - extcacert => $ipa::master::extcacert, - dirsrv_pkcs12 => $ipa::master::dirsrv_pkcs12, - http_pkcs12 => $ipa::master::http_pkcs12, - dirsrv_pin => $ipa::master::dirsrv_pin, - http_pin => $ipa::master::http_pin, - subject => $ipa::master::subject, - selfsign => $ipa::master::selfsign, - require => Ipa::Serverinstall[$::fqdn] - } - } else { - class { 'ipa::service': - require => Ipa::Serverinstall[$::fqdn] - } - } - - ipa::createreplicas { $::fqdn: - } - - firewall { '101 allow IPA master TCP services (http,https,kerberos,kpasswd,ldap,ldaps)': - ensure => 'present', - action => 'accept', - proto => 'tcp', - dport => ['80','88','389','443','464','636'] - } - - firewall { '102 allow IPA master UDP services (kerberos,kpasswd,ntp)': - ensure => 'present', - action => 'accept', - proto => 'udp', - dport => ['88','123','464'] - } - - @@ipa::replicapreparefirewall { $::fqdn: - source => $::ipaddress, - tag => "ipa-replica-prepare-firewall-${ipa::master::domain}" - } - - @@ipa::masterreplicationfirewall { $::fqdn: - source => $::ipaddress, - tag => "ipa-master-replication-firewall-${ipa::master::domain}" - } - - @@ipa::masterprincipal { $::fqdn: - realm => $ipa::master::realm, - tag => "ipa-master-principal-${ipa::master::domain}" - } - - @@ipa::clientinstall { $::fqdn: - masterfqdn => $::fqdn, - domain => $ipa::master::domain, - realm => $ipa::master::realm, - adminpw => $ipa::master::adminpw, - otp => '', - mkhomedir => '', - ntp => '' - } - - if $ipa::master::sudo { - @@ipa::configsudo { $::fqdn: - masterfqdn => $::fqdn, - domain => $ipa::master::domain, - adminpw => $ipa::master::adminpw, - sudopw => $ipa::master::sudopw - } - } - - if $ipa::master::automount { - @@ipa::configautomount { $::fqdn: - masterfqdn => $::fqdn, - os => $::osfamily, - domain => $ipa::master::domain, - realm => $ipa::master::realm - } - } - - if $ipa::master::loadbalance { - ipa::loadbalanceconf { "master-${::fqdn}": - domain => $ipa::master::domain, - ipaservers => $ipa::master::ipaservers, - mkhomedir => $ipa::master::mkhomedir, - require => Ipa::Serverinstall[$::fqdn] - } - } -} diff --git a/ipa/manifests/master_extca.pp b/ipa/manifests/master_extca.pp deleted file mode 100644 index b6f82bd19..000000000 --- a/ipa/manifests/master_extca.pp +++ /dev/null @@ -1,103 +0,0 @@ -# Class: ipa::master_extca -# -# Defines associated files of an IPA server with an external CA -class ipa::master_extca ( - $host = $name, - $realm = {}, - $domain = {}, - $adminpw = {}, - $dspw = {}, - $dnsopt = {}, - $ntpopt = {}, - $extcertpath = {}, - $extcert = {}, - $extcacertpath = {}, - $extcacert = {}, - $dirsrv_pkcs12 = {}, - $http_pkcs12 = {}, - $dirsrv_pin = {}, - $http_pin = {}, - $subject = {}, - $selfsign = {} -) { - - if is_string($extcertpath) and is_string($extcacertpath) { - file { $extcertpath: - ensure => 'file', - owner => 'root', - group => 'root', - mode => '0600', - content => $extcert - } - - file { $extcacertpath: - ensure => 'file', - owner => 'root', - group => 'root', - mode => '0600', - content => $extcacert - } - - if is_string($dirsrv_pkcs12) and is_string($dirsrv_pin) { - $dirsrv_pkcs12opt = "--dirsrv_pkcs12=${dirsrv_pkcs12}" - $dirsrv_pinopt = "--dirsrv_pin=${dirsrv_pin}" - file { $dirsrv_pkcs12: - ensure => 'file', - owner => 'root', - group => 'root', - mode => '0600', - source => "puppet:///files/ipa/${dirsrv_pkcs12}" - } - } else { - $dirsrv_pkcs12opt = '' - $dirsrv_pinopt = '' - } - - if is_string($http_pkcs12) { - $http_pkcs12opt = "--http_pkcs12=${http_pkcs12}" - $http_pinopt = "--http_pin=${http_pin}" - file { $http_pkcs12: - ensure => 'file', - owner => 'root', - group => 'root', - mode => '0600', - source => "puppet:///files/ipa/${http_pkcs12}" - } - } else { - $http_pkcs12opt = '' - $http_pinopt = '' - } - - if is_string($subject) { - $subjectopt = "--subject=${subject}" - } else { - $subjectopt = '' - } - - $selfsignopt = $selfsign ? { - true => '--selfsign', - default => '' - } - - if defined($extcertpath) and defined($extcacertpath) { - if validate_absolute_path($extcertpath) and validate_absolute_path($extcacertpath) { - ipa::serverinstall_extca { $::fqdn: - adminpw => $ipa::master_extca::adminpw, - dspw => $ipa::master_extca::dspw, - extcertpath => $ipa::master_extca::extcertpath, - extcacertpath => $ipa::master_extca::extcacertpath, - dirsrv_pkcs12opt => $ipa::master_extca::dirsrv_pkcs12opt, - http_pkcs12opt => $ipa::master_extca::http_pkcs12opt, - dirsrv_pinopt => $ipa::master_extca::dirsrv_pinopt, - http_pinopt => $ipa::master_extca::http_pinopt, - subjectopt => $ipa::master_extca::subjectopt, - selfsignopt => $ipa::master_extca::selfsignopt - } - - class { 'ipa::service': - require => Ipa::Serverinstall_extca[$::fqdn] - } - } - } - } -} diff --git a/ipa/manifests/masterprincipal.pp b/ipa/manifests/masterprincipal.pp deleted file mode 100644 index 4b078df13..000000000 --- a/ipa/manifests/masterprincipal.pp +++ /dev/null @@ -1,18 +0,0 @@ -define ipa::masterprincipal ( - $host = $name, - $present = true, - $realm = {} -) { - - $principals = suffix(prefix([$host], 'host/'), "@${realm}") - - $ensure = $present ? { - false => 'absent', - default => 'present' - } - - k5login { '/root/.k5login': - ensure => $ensure, - principals => $principals - } -} diff --git a/ipa/manifests/masterreplicationfirewall.pp b/ipa/manifests/masterreplicationfirewall.pp deleted file mode 100644 index 9c0bd771b..000000000 --- a/ipa/manifests/masterreplicationfirewall.pp +++ /dev/null @@ -1,13 +0,0 @@ -define ipa::masterreplicationfirewall ( - $host = $name, - $source = {} -) { - - firewall { "104 allow IPA replication services from master ${host}": - ensure => 'present', - action => 'accept', - proto => 'tcp', - source => $source, - dport => ['9443','9444','9445','7389'] - } -} diff --git a/ipa/manifests/message.pp b/ipa/manifests/message.pp deleted file mode 100644 index f4952337f..000000000 --- a/ipa/manifests/message.pp +++ /dev/null @@ -1,37 +0,0 @@ -# Definition: ipa::message -# -# Sends a notification or a fail message to the log during catalog compilation -define ipa::message ( - $type, - $message -) { - case $type { - 'debug': { - debug($message) - } - 'info': { - info($message) - } - 'alert': { - alert($message) - } - 'crit': { - crit($message) - } - 'emerg': { - emerg($message) - } - 'warning': { - warning($message) - } - 'notice': { - notice($message) - } - 'fail': { - fail($message) - } - default: { - notify { 'Invalid message type, valid types are debug, info, alert, crit, emerg, err, warning, notice or fail': } - } - } -} diff --git a/ipa/manifests/replica.pp b/ipa/manifests/replica.pp deleted file mode 100644 index efb355caa..000000000 --- a/ipa/manifests/replica.pp +++ /dev/null @@ -1,74 +0,0 @@ -# Class: ipa::replica -# -# This class configures an IPA replica -# -# Parameters: -# -# Actions: -# -# Requires: Exported resources, puppetlabs/puppetlabs-firewall, puppetlabs/stdlib -# -# Sample Usage: -# -class ipa::replica ( - $svrpkg = {}, - $adminpw = {}, - $dspw = {}, - $domain = {}, - $kstart = {}, - $sssd = {} -) { - - Class['ipa::client'] -> Ipa::Masterprincipal <<| tag == "ipa-master-principal-${ipa::replica::domain}" |>> -> Ipa::Replicapreparefirewall <<| tag == "ipa-replica-prepare-firewall-${ipa::replica::domain}" |>> -> Ipa::Masterreplicationfirewall <<| tag == "ipa-master-replication-firewall-${ipa::replica::domain}" |>> -> Ipa::Replicainstall[$::fqdn] -> Service['ipa'] - - Ipa::Replicapreparefirewall <<| tag == "ipa-replica-prepare-firewall-${ipa::replica::domain}" |>> - Ipa::Masterreplicationfirewall <<| tag == "ipa-master-replication-firewall-${ipa::replica::domain}" |>> - Ipa::Masterprincipal <<| tag == "ipa-master-principal-${ipa::replica::domain}" |>> - - if $::osfamily != 'RedHat' { - fail("Cannot configure an IPA replica server on ${::operatingsystem} operating systems. Must be a RedHat-like operating system.") - } - - realize Package[$ipa::replica::svrpkg] - - realize Service['ipa'] - - if $ipa::replica::kstart { - realize Package['kstart'] - } - - if $ipa::replica::sssd { - realize Package['sssd-common'] - realize Service['sssd'] - } - - firewall { '101 allow IPA replica TCP services (kerberos,kpasswd,ldap,ldaps)': - ensure => 'present', - action => 'accept', - proto => 'tcp', - dport => ['88','389','464','636'] - } - - firewall { '102 allow IPA replica UDP services (kerberos,kpasswd,ntp)': - ensure => 'present', - action => 'accept', - proto => 'udp', - dport => ['88','123','464'] - } - - ipa::replicainstall { $::fqdn: - adminpw => $ipa::replica::adminpw, - dspw => $ipa::replica::dspw, - require => Package[$ipa::replica::svrpkg] - } - - @@ipa::replicareplicationfirewall { $::fqdn: - source => $::ipaddress, - tag => "ipa-replica-replication-firewall-${ipa::replica::domain}" - } - - @@ipa::replicaprepare { $::fqdn: - dspw => $ipa::replica::dspw, - tag => "ipa-replica-prepare-${ipa::replica::domain}" - } -} diff --git a/ipa/manifests/replicaagreement.pp b/ipa/manifests/replicaagreement.pp deleted file mode 100644 index 225575525..000000000 --- a/ipa/manifests/replicaagreement.pp +++ /dev/null @@ -1,10 +0,0 @@ -define ipa::replicaagreement ( - $from, - $to -) { - exec { "connect-${from}-${to}": - command => "/sbin/runuser -l admin -c \'/usr/sbin/ipa-replica-manage connect ${from} ${to}\'", - unless => "/sbin/runuser -l admin -c \'/usr/sbin/ipa-replica-manage list ${from} | /bin/grep ${to} >/dev/null 2>&1\'", - logoutput => 'on_failure', - } -} diff --git a/ipa/manifests/replicainstall.pp b/ipa/manifests/replicainstall.pp deleted file mode 100644 index a36761021..000000000 --- a/ipa/manifests/replicainstall.pp +++ /dev/null @@ -1,37 +0,0 @@ -# Definition: ipa::replicainstall -# -# Installs an IPA replica -define ipa::replicainstall ( - $host = $name, - $adminpw = {}, - $dspw = {} -) { - - $file = "/var/lib/ipa/replica-info-${host}.gpg" - - Exec["replicainfocheck-${host}"] ~> Exec["clientuninstall-${host}"] ~> Exec["replicainstall-${host}"] ~> Exec["removereplicainfo-${host}"] - - exec { "replicainfocheck-${host}": - command => "/usr/bin/test -e ${file}", - tries => '60', - try_sleep => '60', - unless => '/usr/sbin/ipactl status >/dev/null 2>&1' - } - - exec { "clientuninstall-${host}": - command => '/usr/sbin/ipa-client-install --uninstall --unattended', - refreshonly => true - } - - exec { "replicainstall-${host}": - command => "/usr/sbin/ipa-replica-install --admin-password=${adminpw} --password=${dspw} --skip-conncheck --unattended ${file}", - timeout => '0', - logoutput => 'on_failure', - refreshonly => true - } - - exec { "removereplicainfo-${host}": - command => "/bin/rm -f ${file}", - refreshonly => true - } -} diff --git a/ipa/manifests/replicaprepare.pp b/ipa/manifests/replicaprepare.pp deleted file mode 100644 index 6d736c5b2..000000000 --- a/ipa/manifests/replicaprepare.pp +++ /dev/null @@ -1,30 +0,0 @@ -define ipa::replicaprepare ( - $host = $name, - $dspw = {} -) { - - Cron['k5start_root'] -> Exec["replicaprepare-${host}"] ~> Exec["replica-info-scp-${host}"] ~> Ipa::Hostdelete[$host] - - $file = "/var/lib/ipa/replica-info-${host}.gpg" - - realize Cron['k5start_root'] - - $replicapreparecmd = shellquote('/usr/sbin/ipa-replica-prepare',"--password=${dspw}") - $replicamanagecmd = shellquote('/usr/sbin/ipa-replica-manage',"--password=${dspw}") - - exec { "replicaprepare-${host}": - command => "${replicapreparecmd} ${host}", - unless => "${replicamanagecmd} list | /bin/grep ${host} >/dev/null 2>&1", - timeout => '0' - } - - exec { "replica-info-scp-${host}": - command => shellquote('/usr/bin/scp','-q','-o','StrictHostKeyChecking=no','-o','GSSAPIAuthentication=yes','-o','ConnectTimeout=5','-o','ServerAliveInterval=2',$file,"root@${host}:${file}"), - refreshonly => true, - tries => '60', - try_sleep => '60' - } - - ipa::hostdelete { $host: - } -} diff --git a/ipa/manifests/replicapreparefirewall.pp b/ipa/manifests/replicapreparefirewall.pp deleted file mode 100644 index 09900b3fd..000000000 --- a/ipa/manifests/replicapreparefirewall.pp +++ /dev/null @@ -1,13 +0,0 @@ -define ipa::replicapreparefirewall ( - $host = $name, - $source = {} -) { - - firewall { "103 allow SSH from IPA master ${host}": - ensure => 'present', - action => 'accept', - proto => 'tcp', - source => $source, - dport => ['22'] - } -} diff --git a/ipa/manifests/replicareplicationfirewall.pp b/ipa/manifests/replicareplicationfirewall.pp deleted file mode 100644 index 56f80643d..000000000 --- a/ipa/manifests/replicareplicationfirewall.pp +++ /dev/null @@ -1,13 +0,0 @@ -define ipa::replicareplicationfirewall ( - $host = $name, - $source = {} -) { - - firewall { "104 allow IPA replication services from replica ${host}": - ensure => 'present', - action => 'accept', - proto => 'tcp', - source => $source, - dport => ['9443','9444','9445','7389'] - } -} diff --git a/ipa/manifests/serverinstall.pp b/ipa/manifests/serverinstall.pp deleted file mode 100644 index 5edfbf69a..000000000 --- a/ipa/manifests/serverinstall.pp +++ /dev/null @@ -1,45 +0,0 @@ -# Definition: ipa::serverinstall -# -# Installs an IPA server -define ipa::serverinstall ( - $host = $name, - $realm = {}, - $domain = {}, - $adminpw = {}, - $dspw = {}, - $dnsopt = {}, - $forwarderopts = {}, - $ntpopt = {}, - $extcaopt = {}, - $idstart = {} -) { - - $idstartopt = "--idstart=${idstart}" - - anchor { 'ipa::serverinstall::start': } - - exec { "serverinstall-${host}": - command => "/usr/sbin/ipa-server-install --hostname=${host} --realm=${realm} --domain=${domain} --admin-password=${adminpw} --ds-password=${dspw} ${dnsopt} ${forwarderopts} ${ntpopt} ${extcaopt} ${idstartopt} --unattended", - timeout => '0', - unless => '/usr/sbin/ipactl status >/dev/null 2>&1', - creates => '/etc/ipa/default.conf', - notify => Ipa::Flushcache["server-${host}"], - logoutput => 'on_failure' - } - - ipa::flushcache { "server-${host}": - notify => Ipa::Adminconfig[$host], - require => Anchor['ipa::serverinstall::start'] - } - - ipa::adminconfig { $host: - realm => $realm, - idstart => $idstart, - require => Anchor['ipa::serverinstall::start'] - } - - anchor { 'ipa::serverinstall::end': - require => [Ipa::Flushcache["server-${host}"], Ipa::Adminconfig[$host]] - } - -} diff --git a/ipa/manifests/serverinstall_extca.pp b/ipa/manifests/serverinstall_extca.pp deleted file mode 100644 index c7fa6a1c5..000000000 --- a/ipa/manifests/serverinstall_extca.pp +++ /dev/null @@ -1,29 +0,0 @@ -# Definition: ipa::serverinstall_extca -# -# Completes installation an IPA server with an external CA -define ipa::serverinstall_extca ( - $host = $name, - $adminpw = {}, - $dspw = {}, - $extcertpath = {}, - $extcacertpath = {}, - $dirsrv_pkcs12opt = {}, - $http_pkcs12opt = {}, - $dirsrv_pinopt = {}, - $http_pinopt = {}, - $subjectopt = {}, - $selfsignopt = {} -) { - - exec { "extca_serverinstall-${host}": - command => shellquote('/usr/sbin/ipa-server-install',"--external_cert_file=${extcertpath}","--external_ca_file=${extcacertpath}",$dirsrv_pkcs12opt,$http_pkcs12opt,$dirsrv_pinopt,$http_pinopt,$subjectopt,$selfsignopt,'--unattended'), - timeout => '0', - refreshonly => true, - notify => Ipa::Flushcache["server-${host}"], - require => File[$extcertpath,$extcacertpath], - logoutput => 'on_failure' - } - - ipa::flushcache { "server-${host}": - } -} diff --git a/ipa/manifests/service.pp b/ipa/manifests/service.pp deleted file mode 100644 index 3156e7cb6..000000000 --- a/ipa/manifests/service.pp +++ /dev/null @@ -1,13 +0,0 @@ -# Class: ipa::service -# -# Realizes the IPA service for dependency handling -class ipa::service { - anchor { 'ipa::service::start': } - realize Service['ipa'] - anchor { 'ipa::service::end': } - - Anchor['ipa::service::start'] -> - Service['ipa'] -> - Anchor['ipa::service::end'] - -} diff --git a/ipa/manifests/serviceadd.pp b/ipa/manifests/serviceadd.pp deleted file mode 100644 index 88f48bd90..000000000 --- a/ipa/manifests/serviceadd.pp +++ /dev/null @@ -1,25 +0,0 @@ -define ipa::serviceadd ( - $principal = $name, - $host = undef, - $service = undef, - $realm = undef, -) { - - if $host and $service { - $parsed_principal = $realm ? { - true => "${service}/${host}", - default => "${service}/${host}@${realm}", - } - } else { - $parsed_principal = $principal - } - - Ipa::Hostadd<||> -> Exec["serviceadd-${parsed_principal}"] - - exec { "serviceadd-${parsed_principal}": - command => "/sbin/runuser -l admin -c \'/usr/bin/ipa service-add ${parsed_principal} --force \'", - unless => "/sbin/runuser -l admin -c \'/usr/bin/ipa service-show ${parsed_principal} >/dev/null 2>&1\'", - tries => '60', - try_sleep => '60', - } -} diff --git a/ipa/spec/classes/ipa_spec.rb b/ipa/spec/classes/ipa_spec.rb deleted file mode 100644 index 9ea498f88..000000000 --- a/ipa/spec/classes/ipa_spec.rb +++ /dev/null @@ -1,24 +0,0 @@ -require 'spec_helper' - -describe 'ipa', :type => :class do - describe "on RedHat platform" do - let(:facts) { { :osfamily => 'RedHat' } } - - context 'with master => true' do - describe "ipa" do - let(:params) { - { - :master => true, - :cleanup => false, - :adminpw => '12345678', - :dspw => '12345678', - :domain => 'test.domain.org', - :realm => 'TEST.DOMAIN.ORG' - } - } - it { should contain_class('ipa::master') } - it { should contain_package('ipa-server') } - end - end - end -end diff --git a/ipa/spec/spec_helper.rb b/ipa/spec/spec_helper.rb deleted file mode 100644 index aff774c61..000000000 --- a/ipa/spec/spec_helper.rb +++ /dev/null @@ -1,4 +0,0 @@ -require 'puppet' -require 'rubygems' -require 'rspec-puppet' -require 'puppetlabs_spec_helper/module_spec_helper' diff --git a/ipa/templates/autofs_ldap_auth.conf.erb b/ipa/templates/autofs_ldap_auth.conf.erb deleted file mode 100644 index e7299e073..000000000 --- a/ipa/templates/autofs_ldap_auth.conf.erb +++ /dev/null @@ -1,7 +0,0 @@ - diff --git a/ipa/templates/sudo-ldap.conf.erb b/ipa/templates/sudo-ldap.conf.erb deleted file mode 100644 index 947c3f1bb..000000000 --- a/ipa/templates/sudo-ldap.conf.erb +++ /dev/null @@ -1,12 +0,0 @@ -binddn uid=sudo,cn=sysaccounts,cn=etc,<%= @dc %> -bindpw <%= @sudopw %> - -ssl start_tls -tls_cacertfile /etc/ipa/ca.crt -tls_checkpeer yes - -bind_timelimit 5 -timelimit 15 - -uri ldap://<%= @masterfqdn %> -sudoers_base ou=sudoers,<%= @dc %> diff --git a/ipa/tests/init.pp b/ipa/tests/init.pp deleted file mode 100644 index 4a18caed6..000000000 --- a/ipa/tests/init.pp +++ /dev/null @@ -1,11 +0,0 @@ -# The baseline for module testing used by Puppet Labs is that each manifest -# should have a corresponding test manifest that declares that class or defined -# type. -# -# Tests are then run by using puppet apply --noop (to check for compilation errors -# and view a log of events) or by fully applying the test in a virtual environment -# (to compare the resulting system state to the desired state). -# -# Learn more about module testing here: http://docs.puppetlabs.com/guides/tests_smoke.html -# -include ipa From 779bd15b828e68b99327ebd105cfa25db75dc4b2 Mon Sep 17 00:00:00 2001 From: Gael Chamoulaud Date: Thu, 4 Feb 2016 15:25:47 +0100 Subject: [PATCH 23/35] Add camptocamp/puppet-kmod This module update commit was generated by Bade. For more info please check https://github.com/paramite/bade This commit is setting modules to following state: kmod - initial commit: ea03df0eff7b7e5faccb9c4e386d451301468f04 Change-Id: Idcf3505c54a800e2085c97ebb05f8bf5ed9d2da9 Signed-off-by: Gael Chamoulaud --- Puppetfile | 4 + kmod/.fixtures.yml | 5 + kmod/.gitignore | 10 + kmod/.puppet-lint.rc | 5 + kmod/.sync.yml | 3 + kmod/.travis.yml | 32 +++ kmod/CHANGELOG.md | 81 +++++++ kmod/Gemfile | 47 ++++ kmod/LICENSE | 201 ++++++++++++++++++ kmod/README.md | 116 ++++++++++ kmod/Rakefile | 23 ++ kmod/manifests/alias.pp | 26 +++ kmod/manifests/blacklist.pp | 27 +++ kmod/manifests/init.pp | 21 ++ kmod/manifests/install.pp | 30 +++ kmod/manifests/load.pp | 87 ++++++++ kmod/manifests/option.pp | 35 +++ kmod/manifests/setting.pp | 45 ++++ kmod/metadata.json | 44 ++++ .../nodesets/centos-5-x86_64-docker.yml | 15 ++ .../nodesets/centos-6-x86_64-docker.yml | 15 ++ .../nodesets/centos-6-x86_64-openstack.yml | 14 ++ .../nodesets/centos-6-x86_64-vagrant.yml | 11 + .../nodesets/centos-7-x86_64-docker.yml | 15 ++ .../nodesets/centos-7-x86_64-openstack.yml | 14 ++ .../nodesets/centos-7-x86_64-vagrant.yml | 11 + .../nodesets/debian-6-x86_64-docker.yml | 15 ++ .../nodesets/debian-6-x86_64-openstack.yml | 14 ++ .../nodesets/debian-6-x86_64-vagrant.yml | 11 + .../nodesets/debian-7-x86_64-docker.yml | 15 ++ .../nodesets/debian-7-x86_64-openstack.yml | 14 ++ .../nodesets/debian-7-x86_64-vagrant.yml | 11 + .../nodesets/debian-8-x86_64-docker.yml | 15 ++ .../nodesets/debian-8-x86_64-openstack.yml | 14 ++ .../nodesets/debian-8-x86_64-vagrant.yml | 11 + .../nodesets/ubuntu-10.04-x86_64-docker.yml | 13 ++ .../nodesets/ubuntu-12.04-x86_64-docker.yml | 15 ++ .../ubuntu-12.04-x86_64-openstack.yml | 14 ++ .../nodesets/ubuntu-14.04-x86_64-docker.yml | 15 ++ .../ubuntu-14.04-x86_64-openstack.yml | 14 ++ .../nodesets/ubuntu-14.04-x86_64-vagrant.yml | 11 + .../nodesets/ubuntu-14.10-x86_64-docker.yml | 15 ++ .../ubuntu-14.10-x86_64-openstack.yml | 14 ++ .../nodesets/ubuntu-15.04-x86_64-docker.yml | 15 ++ .../ubuntu-15.04-x86_64-openstack.yml | 14 ++ kmod/spec/classes/kmod_spec.rb | 26 +++ kmod/spec/defines/kmod_alias_spec.rb | 42 ++++ kmod/spec/defines/kmod_blacklist_spec.rb | 49 +++++ kmod/spec/defines/kmod_install_spec.rb | 26 +++ kmod/spec/defines/kmod_load_spec.rb | 67 ++++++ kmod/spec/defines/kmod_setting_spec.rb | 40 ++++ kmod/spec/spec.opts | 6 + kmod/spec/spec_helper.rb | 42 ++++ kmod/templates/redhat.modprobe.erb | 5 + 54 files changed, 1500 insertions(+) create mode 100644 kmod/.fixtures.yml create mode 100644 kmod/.gitignore create mode 100644 kmod/.puppet-lint.rc create mode 100644 kmod/.sync.yml create mode 100644 kmod/.travis.yml create mode 100644 kmod/CHANGELOG.md create mode 100644 kmod/Gemfile create mode 100644 kmod/LICENSE create mode 100644 kmod/README.md create mode 100644 kmod/Rakefile create mode 100644 kmod/manifests/alias.pp create mode 100644 kmod/manifests/blacklist.pp create mode 100644 kmod/manifests/init.pp create mode 100644 kmod/manifests/install.pp create mode 100644 kmod/manifests/load.pp create mode 100644 kmod/manifests/option.pp create mode 100644 kmod/manifests/setting.pp create mode 100644 kmod/metadata.json create mode 100644 kmod/spec/acceptance/nodesets/centos-5-x86_64-docker.yml create mode 100644 kmod/spec/acceptance/nodesets/centos-6-x86_64-docker.yml create mode 100644 kmod/spec/acceptance/nodesets/centos-6-x86_64-openstack.yml create mode 100644 kmod/spec/acceptance/nodesets/centos-6-x86_64-vagrant.yml create mode 100644 kmod/spec/acceptance/nodesets/centos-7-x86_64-docker.yml create mode 100644 kmod/spec/acceptance/nodesets/centos-7-x86_64-openstack.yml create mode 100644 kmod/spec/acceptance/nodesets/centos-7-x86_64-vagrant.yml create mode 100644 kmod/spec/acceptance/nodesets/debian-6-x86_64-docker.yml create mode 100644 kmod/spec/acceptance/nodesets/debian-6-x86_64-openstack.yml create mode 100644 kmod/spec/acceptance/nodesets/debian-6-x86_64-vagrant.yml create mode 100644 kmod/spec/acceptance/nodesets/debian-7-x86_64-docker.yml create mode 100644 kmod/spec/acceptance/nodesets/debian-7-x86_64-openstack.yml create mode 100644 kmod/spec/acceptance/nodesets/debian-7-x86_64-vagrant.yml create mode 100644 kmod/spec/acceptance/nodesets/debian-8-x86_64-docker.yml create mode 100644 kmod/spec/acceptance/nodesets/debian-8-x86_64-openstack.yml create mode 100644 kmod/spec/acceptance/nodesets/debian-8-x86_64-vagrant.yml create mode 100644 kmod/spec/acceptance/nodesets/ubuntu-10.04-x86_64-docker.yml create mode 100644 kmod/spec/acceptance/nodesets/ubuntu-12.04-x86_64-docker.yml create mode 100644 kmod/spec/acceptance/nodesets/ubuntu-12.04-x86_64-openstack.yml create mode 100644 kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-docker.yml create mode 100644 kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-openstack.yml create mode 100644 kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-vagrant.yml create mode 100644 kmod/spec/acceptance/nodesets/ubuntu-14.10-x86_64-docker.yml create mode 100644 kmod/spec/acceptance/nodesets/ubuntu-14.10-x86_64-openstack.yml create mode 100644 kmod/spec/acceptance/nodesets/ubuntu-15.04-x86_64-docker.yml create mode 100644 kmod/spec/acceptance/nodesets/ubuntu-15.04-x86_64-openstack.yml create mode 100644 kmod/spec/classes/kmod_spec.rb create mode 100644 kmod/spec/defines/kmod_alias_spec.rb create mode 100644 kmod/spec/defines/kmod_blacklist_spec.rb create mode 100644 kmod/spec/defines/kmod_install_spec.rb create mode 100644 kmod/spec/defines/kmod_load_spec.rb create mode 100644 kmod/spec/defines/kmod_setting_spec.rb create mode 100644 kmod/spec/spec.opts create mode 100644 kmod/spec/spec_helper.rb create mode 100644 kmod/templates/redhat.modprobe.erb diff --git a/Puppetfile b/Puppetfile index 92d707ab0..97f05290d 100644 --- a/Puppetfile +++ b/Puppetfile @@ -118,6 +118,10 @@ mod 'kibana3', :commit => '6ca9631fbe82766134f98e2e8780bb91e7cd3f0e', :git => 'https://github.com/thejandroman/puppet-kibana3.git' +mod 'kmod', + :commit => 'ea03df0eff7b7e5faccb9c4e386d451301468f04', + :git => 'https://github.com/camptocamp/puppet-kmod.git' + mod 'manila', :commit => '9d74004a388f65d8e54c8ab4dfb5832a3b024925', :git => 'https://github.com/openstack/puppet-manila.git' diff --git a/kmod/.fixtures.yml b/kmod/.fixtures.yml new file mode 100644 index 000000000..f172a1973 --- /dev/null +++ b/kmod/.fixtures.yml @@ -0,0 +1,5 @@ +fixtures: + repositories: + "stdlib": "git://github.com/puppetlabs/puppetlabs-stdlib.git" + symlinks: + "kmod": "#{source_dir}" diff --git a/kmod/.gitignore b/kmod/.gitignore new file mode 100644 index 000000000..65839fa01 --- /dev/null +++ b/kmod/.gitignore @@ -0,0 +1,10 @@ +pkg/ +Gemfile.lock +vendor/ +spec/fixtures/ +.vagrant/ +.bundle/ +coverage/ +log/ +.*.swp +*~ diff --git a/kmod/.puppet-lint.rc b/kmod/.puppet-lint.rc new file mode 100644 index 000000000..d8f5c59ef --- /dev/null +++ b/kmod/.puppet-lint.rc @@ -0,0 +1,5 @@ +--fail-on-warnings +--relative +--no-80chars +--no-documentation +--no-class_inherits_from_params_class-check diff --git a/kmod/.sync.yml b/kmod/.sync.yml new file mode 100644 index 000000000..d62948585 --- /dev/null +++ b/kmod/.sync.yml @@ -0,0 +1,3 @@ +--- +.travis.yml: + forge_password: "QvaBWqv7ZaB34BzONHPxZy+1OW+eBP4dZmhDobAVFD20kOP4O4QHdCe9kS3y3JRXrNW3wuiuegNzoKb+JvqQrZPQF3Ea0DO4uCPVrnVbrnmGC0p7v2VDA1tsb2snnSMa5mC6+3wYMwS27zRSQ7vN3AwDKMdp/jcPbTEpnv/+314=" diff --git a/kmod/.travis.yml b/kmod/.travis.yml new file mode 100644 index 000000000..e5397c5b7 --- /dev/null +++ b/kmod/.travis.yml @@ -0,0 +1,32 @@ +--- +language: ruby +sudo: false +cache: bundler +bundler_args: --without system_tests +script: ["bundle exec rake validate", "bundle exec rake lint", "bundle exec rake spec SPEC_OPTS='--format documentation'", "bundle exec rake metadata"] +matrix: + fast_finish: true + include: + - rvm: 1.8.7 + env: PUPPET_GEM_VERSION="~> 3.0" FACTER_GEM_VERSION="~> 1.7.0" + - rvm: 1.9.3 + env: PUPPET_GEM_VERSION="~> 3.0" + - rvm: 2.0.0 + env: PUPPET_GEM_VERSION="~> 3.0" + - rvm: 2.0.0 + env: PUPPET_GEM_VERSION="~> 3.0" FUTURE_PARSER="yes" + - rvm: 2.1.6 + env: PUPPET_GEM_VERSION="~> 4.0" +notifications: + email: false +deploy: + provider: puppetforge + user: camptocamp + password: + secure: "QvaBWqv7ZaB34BzONHPxZy+1OW+eBP4dZmhDobAVFD20kOP4O4QHdCe9kS3y3JRXrNW3wuiuegNzoKb+JvqQrZPQF3Ea0DO4uCPVrnVbrnmGC0p7v2VDA1tsb2snnSMa5mC6+3wYMwS27zRSQ7vN3AwDKMdp/jcPbTEpnv/+314=" + on: + tags: true + # all_branches is required to use tags + all_branches: true + # Only publish if our main Ruby target builds + rvm: 1.9.3 diff --git a/kmod/CHANGELOG.md b/kmod/CHANGELOG.md new file mode 100644 index 000000000..54daf6d69 --- /dev/null +++ b/kmod/CHANGELOG.md @@ -0,0 +1,81 @@ +## 2015-08-27 - Release 2.1.0 + +Add minimal SuSE support + +## 2015-08-21 - Release 2.0.11 + +Use docker for acceptance tests + +## 2015-06-26 - Release 2.0.10 + +Fix strict_variables activation with rspec-puppet 2.2 + +## 2015-05-28 - Release 2.0.9 + +Add beaker_spec_helper to Gemfile + +## 2015-05-26 - Release 2.0.8 + +Use random application order in nodeset + +## 2015-05-26 - Release 2.0.7 + +add utopic & vivid nodesets + +## 2015-05-25 - Release 2.0.6 + +Don't allow failure on Puppet 4 + +## 2015-05-13 - Release 2.0.5 + +Add puppet-lint-file_source_rights-check gem + +## 2015-05-12 - Release 2.0.4 + +Don't pin beaker + +## 2015-04-27 - Release 2.0.3 + +Add nodeset ubuntu-12.04-x86_64-openstack + +## 2015-04-18 - Release 2.0.2 + +- Add beaker nodeset + +## 2015-04-15 - Release 2.0.1 + +- Fix kmod::install's file class parameter's default + +## 2015-04-03 - Release 2.0.0 + +- Add kmod::option and refactored everything to use kmod::setting +- removed obsolete generic.pp + +## 2015-03-24 - Release 1.0.6 + +- Lint + +## 2015-01-19 - Release 1.0.5 + +- Fix relative class inclusions + +## 2015-01-07 - Release 1.0.4 + +- Fix unquoted strings in cases + +## 2014-12-16 - Release 1.0.1 + +- Fix for future parser + +## 2014-10-20 - Release 1.0.0 + +- Setup automatic Forge releases + +## 2014-07-02 - Release 0.1.1 + +- Fix deprecation warnings, #22 + +## 2014-07-02 - Release 0.1.0 + +- Add unit tests +- Various improvements diff --git a/kmod/Gemfile b/kmod/Gemfile new file mode 100644 index 000000000..0cb59337f --- /dev/null +++ b/kmod/Gemfile @@ -0,0 +1,47 @@ +source ENV['GEM_SOURCE'] || "https://rubygems.org" + +group :development, :unit_tests do + gem 'rake', :require => false + gem 'rspec', '< 3.2', :require => false if RUBY_VERSION =~ /^1.8/ + gem 'rspec-puppet', :require => false + gem 'puppetlabs_spec_helper', :require => false + gem 'metadata-json-lint', :require => false + gem 'puppet-lint', :require => false + gem 'puppet-lint-unquoted_string-check', :require => false + gem 'puppet-lint-empty_string-check', :require => false + gem 'puppet-lint-spaceship_operator_without_tag-check', :require => false + gem 'puppet-lint-variable_contains_upcase', :require => false + gem 'puppet-lint-absolute_classname-check', :require => false + gem 'puppet-lint-undef_in_function-check', :require => false + gem 'puppet-lint-leading_zero-check', :require => false + gem 'puppet-lint-trailing_comma-check', :require => false + gem 'puppet-lint-file_ensure-check', :require => false + gem 'puppet-lint-version_comparison-check', :require => false + gem 'puppet-lint-fileserver-check', :require => false + gem 'puppet-lint-file_source_rights-check', :require => false + gem 'puppet-lint-alias-check', :require => false + gem 'rspec-puppet-facts', :require => false + gem 'github_changelog_generator', :require => false, :git => 'https://github.com/raphink/github-changelog-generator.git', :branch => 'dev/all_patches' if RUBY_VERSION !~ /^1.8/ + gem 'puppet-blacksmith', :require => false if RUBY_VERSION !~ /^1.8/ +end + +group :system_tests do + gem 'beaker', :require => false + gem 'beaker-rspec', :require => false + gem 'beaker_spec_helper', :require => false + gem 'serverspec', :require => false +end + +if facterversion = ENV['FACTER_GEM_VERSION'] + gem 'facter', facterversion, :require => false +else + gem 'facter', :require => false +end + +if puppetversion = ENV['PUPPET_GEM_VERSION'] + gem 'puppet', puppetversion, :require => false +else + gem 'puppet', :require => false +end + +# vim:ft=ruby diff --git a/kmod/LICENSE b/kmod/LICENSE new file mode 100644 index 000000000..8d968b6cb --- /dev/null +++ b/kmod/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/kmod/README.md b/kmod/README.md new file mode 100644 index 000000000..5b378dbae --- /dev/null +++ b/kmod/README.md @@ -0,0 +1,116 @@ +# Kmod Puppet module + +[![Puppet Forge Version](http://img.shields.io/puppetforge/v/camptocamp/kmod.svg)](https://forge.puppetlabs.com/camptocamp/kmod) +[![Puppet Forge Downloads](http://img.shields.io/puppetforge/dt/camptocamp/kmod.svg)](https://forge.puppetlabs.com/camptocamp/kmod) +[![Build Status](https://img.shields.io/travis/camptocamp/puppet-kmod/master.svg)](https://travis-ci.org/camptocamp/puppet-kmod) +[![Puppet Forge Endorsement](https://img.shields.io/puppetforge/e/camptocamp/kmod.svg)](https://forge.puppetlabs.com/camptocamp/kmod) +[![Gemnasium](https://img.shields.io/gemnasium/camptocamp/puppet-kmod.svg)](https://gemnasium.com/camptocamp/puppet-kmod) +[![By Camptocamp](https://img.shields.io/badge/by-camptocamp-fb7047.svg)](http://www.camptocamp.com) + +## Description + +This module provides definitions to manipulate modprobe.conf (5) stanzas: + + * kmod::alias + * kmod::install + * kmod::blacklist + +It depends on Augeas with the modprobe lens. + +## Usage + +This module has five main defined types: + + * kmod::load + * kmod::alias + * kmod::option + * kmod::install + * kmod::blacklist + + +### kmod::load + +Loads a module using modprobe and manages persistent modules in /etc/sysconfig/modules + +```puppet + kmod::load { 'mymodule': } +``` + +### kmod::alias + +Adds an alias to modprobe.conf, by default `/etc/modprobe.d/.conf` is assumed for a filename. + +```puppet + kmod::alias { 'bond0': + modulename => 'bonding', + } +``` + +Params: +* `modulename`: Name of the module to alias +* `aliasname`: Name of the alias (defaults to the resource title) +* `file`: File to write to (see above default) + +### kmod::option + +Adds an option to modprobe.conf + +```puppet + kmod::option { 'bond0 mode': + module => 'bond0', + option => 'mode', + value => '1', + } + + kmod::option { 'bond0': + option => 'mode', + value => '1', + } +``` + +Params: +* `option`: Name of the parameter to add +* `value`: Value of the parameter +* `module`: Name of the module (if ommited, the resource title is used) +* `file`: File to write to (defaults to `/etc/modprobe.d/.conf`) + +### kmod::blacklist + +Manages modprobe blacklist entries. Blacklist entries prevents module aliases from being used, +but would not prevent the module from being loaded. +To prevent a module from being loaded use `kmod::install` + +```puppet + kmod::blacklist { 'foo': } +``` + +Params: +* `file`: File to write to, defaults to `/etc/modprobe.d/blacklist.conf` + +### kmod::install + +Manage modprobe install entries + +```puppet + kmod::install { 'pcspkr': } +``` + +If you want to ensure that module can't be loaded at all you can do the following: +```puppet + kmod::install { 'dccp': command => '/bin/false' } +``` + +Params: +* `file`: File to write to (defaults to `/etc/modprobe.d/.conf`) +* `command`: (optional) command associated with the install, defaults to `/bin/true` + + + +## Contributing + +Please report bugs and feature request using [GitHub issue +tracker](https://github.com/camptocamp/puppet-kmod/issues). + +For pull requests, it is very much appreciated to check your Puppet manifest +with [puppet-lint](https://github.com/camptocamp/puppet-kmod/issues) to follow the recommended Puppet style guidelines from the +[Puppet Labs style guide](http://docs.puppetlabs.com/guides/style_guide.html). diff --git a/kmod/Rakefile b/kmod/Rakefile new file mode 100644 index 000000000..adcac180c --- /dev/null +++ b/kmod/Rakefile @@ -0,0 +1,23 @@ +require 'puppetlabs_spec_helper/rake_tasks' +require 'puppet-lint/tasks/puppet-lint' + +Rake::Task[:lint].clear +PuppetLint::RakeTask.new :lint do |config| + config.ignore_paths = ["spec/**/*.pp", "pkg/**/*.pp", "vendor/**/*.pp"] + config.disable_checks = ['80chars'] + config.fail_on_warnings = true +end + +PuppetSyntax.exclude_paths = ["spec/fixtures/**/*.pp", "vendor/**/*"] + +# Publishing tasks +unless RUBY_VERSION =~ /^1\.8/ + require 'puppet_blacksmith' + require 'puppet_blacksmith/rake_tasks' + require 'github_changelog_generator/task' + GitHubChangelogGenerator::RakeTask.new :changelog do |config| + m = Blacksmith::Modulefile.new + config.future_release = m.version + config.release_url = "https://forge.puppetlabs.com/#{m.author}/#{m.name}/%s" + end +end diff --git a/kmod/manifests/alias.pp b/kmod/manifests/alias.pp new file mode 100644 index 000000000..dcdb974f7 --- /dev/null +++ b/kmod/manifests/alias.pp @@ -0,0 +1,26 @@ +# = Define: kmod::alias +# +# == Example +# +# kmod::alias { 'bond0': +# alias => 'bonding', +# } +# +define kmod::alias( + $source, + $ensure = 'present', + $file = "/etc/modprobe.d/${name}.conf", + $aliasname = $name, +) { + + include ::kmod + + kmod::setting { "kmod::alias ${title}": + module => $aliasname, + file => $file, + category => 'alias', + option => 'modulename', + value => $source, + } + +} diff --git a/kmod/manifests/blacklist.pp b/kmod/manifests/blacklist.pp new file mode 100644 index 000000000..1dddddaa3 --- /dev/null +++ b/kmod/manifests/blacklist.pp @@ -0,0 +1,27 @@ +# +# == Definition: kmod::blacklist +# +# Set a kernel module as blacklisted. +# +# Parameters: +# - *ensure*: present/absent; +# - *file*: optionally, set the file where the stanza is written. +# +# Example usage: +# +# kmod::blacklist { 'pcspkr': } +# +define kmod::blacklist( + $ensure=present, + $file='/etc/modprobe.d/blacklist.conf', +) { + + + kmod::setting { "kmod::blacklist ${title}": + ensure => $ensure, + module => $name, + file => $file, + category => 'blacklist', + } + +} diff --git a/kmod/manifests/init.pp b/kmod/manifests/init.pp new file mode 100644 index 000000000..14da62ba0 --- /dev/null +++ b/kmod/manifests/init.pp @@ -0,0 +1,21 @@ +# +# == Class: kmod +# +# Ensures a couple of mandatory files are present before managing their +# content. +# +# +class kmod { + + if versioncmp($::augeasversion, '0.9.0') < 0 { + fail('Augeas 0.10.0 or higher required') + } + file { '/etc/modprobe.d': ensure => directory } + + file { [ + '/etc/modprobe.d/modprobe.conf', + '/etc/modprobe.d/aliases.conf', + '/etc/modprobe.d/blacklist.conf', + ]: ensure => file, + } +} diff --git a/kmod/manifests/install.pp b/kmod/manifests/install.pp new file mode 100644 index 000000000..8f6bb74c0 --- /dev/null +++ b/kmod/manifests/install.pp @@ -0,0 +1,30 @@ +# +# == Definition: kmod::install +# +# Set a kernel module as installed. +# +# Parameters: +# - *ensure*: present/absent; +# - *command*: optionally, set the command associated with the kernel module; +# - *file*: optionally, set the file where the stanza is written. +# +# Example usage: +# +# kmod::install { 'pcspkr': } +# +define kmod::install( + $ensure=present, + $command='/bin/true', + $file="/etc/modprobe.d/${name}.conf", +) { + + kmod::setting { "kmod::install ${title}": + ensure => $ensure, + module => $name, + file => $file, + category => 'install', + option => 'command', + value => $command, + } + +} diff --git a/kmod/manifests/load.pp b/kmod/manifests/load.pp new file mode 100644 index 000000000..07525681b --- /dev/null +++ b/kmod/manifests/load.pp @@ -0,0 +1,87 @@ +# +# == Definition: kmod::load +# +# Manage a kernel module in /etc/modules. +# +# Parameters: +# - *ensure*: present/absent; +# - *file*: optionally, set the file where the stanza is written. +# +# Example usage: +# +# kmod::load { 'sha256': } +# +define kmod::load( + $ensure=present, + $file='/etc/modules', +) { + + case $ensure { + 'present': { + case $::osfamily { + 'Debian': { + $changes = "clear '${name}'" + } + 'Suse': { + $changes = "set MODULES_LOADED_ON_BOOT/value[.='${name}'] '${name}'" + } + default: { } + } + + exec { "modprobe ${name}": + path => '/bin:/sbin:/usr/bin:/usr/sbin', + unless => "egrep -q '^${name} ' /proc/modules", + } + } + + 'absent': { + case $::osfamily { + 'Debian': { + $changes = "rm '${name}'" + } + 'Suse': { + $changes = "rm MODULES_LOADED_ON_BOOT/value[.='${name}']" + } + default: { } + } + + exec { "modprobe -r ${name}": + path => '/bin:/sbin:/usr/bin:/usr/sbin', + onlyif => "egrep -q '^${name} ' /proc/modules", + } + } + + default: { fail "${module_name}: unknown ensure value ${ensure}" } + } + + case $::osfamily { + 'Debian': { + augeas {"Manage ${name} in ${file}": + incl => $file, + lens => 'Modules.lns', + changes => $changes, + } + } + 'RedHat': { + file { "/etc/sysconfig/modules/${name}.modules": + ensure => $ensure, + mode => '0755', + content => template('kmod/redhat.modprobe.erb'), + } + } + 'Suse': { + $kernelfile = $file ? { + '/etc/modules' => '/etc/sysconfig/kernel', + default => $file, + } + augeas { "sysconfig_kernel_MODULES_LOADED_ON_BOOT_${name}": + lens => 'Shellvars_list.lns', + incl => $kernelfile, + changes => $changes, + } + } + default: { + fail "${module_name}: Unknown OS family ${::osfamily}" + } + } +} diff --git a/kmod/manifests/option.pp b/kmod/manifests/option.pp new file mode 100644 index 000000000..8597138f6 --- /dev/null +++ b/kmod/manifests/option.pp @@ -0,0 +1,35 @@ +# = Define: kmod::alias +# +# == Example +# +# kmod::option { 'bond0': +# option => 'bonding', +# } +# +define kmod::option( + $option, + $value, + $module = $name, + $ensure = 'present', + $file = undef, +) { + + include ::kmod + + $target_file = $file ? { + undef => "/etc/modprobe.d/${module}.conf", + default => $file, + } + + + kmod::setting { "kmod::option ${title}": + ensure => $ensure, + module => $module, + category => 'options', + file => $target_file, + option => $option, + value => $value, + } + +} + diff --git a/kmod/manifests/setting.pp b/kmod/manifests/setting.pp new file mode 100644 index 000000000..b554d1646 --- /dev/null +++ b/kmod/manifests/setting.pp @@ -0,0 +1,45 @@ +# = Define: kmod::setting +# +# == Example +# +# +define kmod::setting( + $file, + $category, + $option = undef, + $value = undef, + $module = $name, + $ensure = 'present', +) { + + include ::kmod + + ensure_resource('file', $file, { 'ensure' => 'file'} ) + case $ensure { + 'present': { + if $option { + $changes = [ + "set ${category}[. = '${module}'] ${module}", + "set ${category}[. = '${module}']/${option} ${value}", + ] + } else { + $changes = [ + "set ${category}[. = '${module}'] ${module}", + ] + } + } + + 'absent': { + $changes = "rm ${category}[. = '${module}']" + } + + default: { fail ( "unknown ensure value ${ensure}" ) } + } + + augeas { "kmod::setting ${title} ${module}": + incl => $file, + lens => 'Modprobe.lns', + changes => $changes, + require => File[$file], + } +} diff --git a/kmod/metadata.json b/kmod/metadata.json new file mode 100644 index 000000000..25cb4b183 --- /dev/null +++ b/kmod/metadata.json @@ -0,0 +1,44 @@ +{ + "name": "camptocamp-kmod", + "version": "2.1.0", + "author": "camptocamp", + "summary": "Manage Linux kernel modules with Puppet", + "license": "Apache-2.0", + "source": "https://github.com/camptocamp/puppet-kmod", + "project_page": "https://github.com/camptocamp/puppet-kmod", + "issues_url": "https://github.com/camptocamp/puppet-kmod/issues", + "description": "Manage Linux kernel modules with Puppet", + "dependencies": [ + + ], + "operatingsystem_support": [ + { + "operatingsystem": "Debian", + "operatingsystemrelease": [ + "6", + "7" + ] + }, + { + "operatingsystem": "Ubuntu", + "operatingsystemrelease": [ + "12.04", + "14.04" + ] + }, + { + "operatingsystem": "SLES", + "operatingsystemrelease": [ + "11" + ] + }, + { + "operatingsystem": "RedHat", + "operatingsystemrelease": [ + "5", + "6", + "7" + ] + } + ] +} diff --git a/kmod/spec/acceptance/nodesets/centos-5-x86_64-docker.yml b/kmod/spec/acceptance/nodesets/centos-5-x86_64-docker.yml new file mode 100644 index 000000000..679afb04d --- /dev/null +++ b/kmod/spec/acceptance/nodesets/centos-5-x86_64-docker.yml @@ -0,0 +1,15 @@ +HOSTS: + centos-5-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-5-x86_64 + hypervisor : docker + image: centos:5 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'yum install -y crontabs tar wget' +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/centos-6-x86_64-docker.yml b/kmod/spec/acceptance/nodesets/centos-6-x86_64-docker.yml new file mode 100644 index 000000000..9cab03d08 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/centos-6-x86_64-docker.yml @@ -0,0 +1,15 @@ +HOSTS: + centos-6-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-6-x86_64 + hypervisor : docker + image: centos:6 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'yum install -y crontabs tar wget' +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/centos-6-x86_64-openstack.yml b/kmod/spec/acceptance/nodesets/centos-6-x86_64-openstack.yml new file mode 100644 index 000000000..e325b9e90 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/centos-6-x86_64-openstack.yml @@ -0,0 +1,14 @@ +HOSTS: + centos-6-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-6-x86_64 + hypervisor : openstack + flavor: m1.small + image: centos-6-latest + user: root +CONFIG: + type: foss + log_level: debug + openstack_network: default diff --git a/kmod/spec/acceptance/nodesets/centos-6-x86_64-vagrant.yml b/kmod/spec/acceptance/nodesets/centos-6-x86_64-vagrant.yml new file mode 100644 index 000000000..f06036ecc --- /dev/null +++ b/kmod/spec/acceptance/nodesets/centos-6-x86_64-vagrant.yml @@ -0,0 +1,11 @@ +HOSTS: + centos-6-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-6-x86_64 + hypervisor : vagrant + box : camptocamp/centos-6-x86_64 +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/centos-7-x86_64-docker.yml b/kmod/spec/acceptance/nodesets/centos-7-x86_64-docker.yml new file mode 100644 index 000000000..0bc972711 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/centos-7-x86_64-docker.yml @@ -0,0 +1,15 @@ +HOSTS: + centos-7-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-7-x86_64 + hypervisor : docker + image: centos:7 + docker_preserve_image: true + docker_cmd: '["/usr/sbin/init"]' + docker_image_commands: + - 'yum install -y crontabs tar wget' +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/centos-7-x86_64-openstack.yml b/kmod/spec/acceptance/nodesets/centos-7-x86_64-openstack.yml new file mode 100644 index 000000000..9003c8678 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/centos-7-x86_64-openstack.yml @@ -0,0 +1,14 @@ +HOSTS: + centos-7-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-7-x86_64 + hypervisor : openstack + flavor: m1.small + image: centos-7-latest + user: centos +CONFIG: + type: foss + log_level: debug + openstack_network: default diff --git a/kmod/spec/acceptance/nodesets/centos-7-x86_64-vagrant.yml b/kmod/spec/acceptance/nodesets/centos-7-x86_64-vagrant.yml new file mode 100644 index 000000000..95402e546 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/centos-7-x86_64-vagrant.yml @@ -0,0 +1,11 @@ +HOSTS: + centos-7-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-7-x86_64 + hypervisor : vagrant + box : camptocamp/centos-7-x86_64 +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/debian-6-x86_64-docker.yml b/kmod/spec/acceptance/nodesets/debian-6-x86_64-docker.yml new file mode 100644 index 000000000..359dae7dc --- /dev/null +++ b/kmod/spec/acceptance/nodesets/debian-6-x86_64-docker.yml @@ -0,0 +1,15 @@ +HOSTS: + debian-6-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-6-amd64 + hypervisor : docker + image: debian:6 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y wget' +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/debian-6-x86_64-openstack.yml b/kmod/spec/acceptance/nodesets/debian-6-x86_64-openstack.yml new file mode 100644 index 000000000..c6c192fe4 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/debian-6-x86_64-openstack.yml @@ -0,0 +1,14 @@ +HOSTS: + debian-6-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-6-amd64 + hypervisor : openstack + flavor: m1.small + image: debian-6-latest + user: debian +CONFIG: + type: foss + log_level: debug + openstack_network: default diff --git a/kmod/spec/acceptance/nodesets/debian-6-x86_64-vagrant.yml b/kmod/spec/acceptance/nodesets/debian-6-x86_64-vagrant.yml new file mode 100644 index 000000000..03db0fa76 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/debian-6-x86_64-vagrant.yml @@ -0,0 +1,11 @@ +HOSTS: + debian-6-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-6-amd64 + hypervisor : vagrant + box : puppetlabs/debian-6.0.10-64-nocm +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/debian-7-x86_64-docker.yml b/kmod/spec/acceptance/nodesets/debian-7-x86_64-docker.yml new file mode 100644 index 000000000..fc11f5748 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/debian-7-x86_64-docker.yml @@ -0,0 +1,15 @@ +HOSTS: + debian-7-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-7-amd64 + hypervisor : docker + image: debian:7 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y cron wget' +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/debian-7-x86_64-openstack.yml b/kmod/spec/acceptance/nodesets/debian-7-x86_64-openstack.yml new file mode 100644 index 000000000..017b4c743 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/debian-7-x86_64-openstack.yml @@ -0,0 +1,14 @@ +HOSTS: + debian-7-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-7-amd64 + hypervisor : openstack + flavor: m1.small + image: debian-7-latest + user: debian +CONFIG: + type: foss + log_level: debug + openstack_network: default diff --git a/kmod/spec/acceptance/nodesets/debian-7-x86_64-vagrant.yml b/kmod/spec/acceptance/nodesets/debian-7-x86_64-vagrant.yml new file mode 100644 index 000000000..8ed1264df --- /dev/null +++ b/kmod/spec/acceptance/nodesets/debian-7-x86_64-vagrant.yml @@ -0,0 +1,11 @@ +HOSTS: + debian-7-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-7-amd64 + hypervisor : vagrant + box : camptocamp/debian-7-amd64 +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/debian-8-x86_64-docker.yml b/kmod/spec/acceptance/nodesets/debian-8-x86_64-docker.yml new file mode 100644 index 000000000..86a55e156 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/debian-8-x86_64-docker.yml @@ -0,0 +1,15 @@ +HOSTS: + debian-8-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-8-amd64 + hypervisor : docker + image: debian:8 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y cron wget' +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/debian-8-x86_64-openstack.yml b/kmod/spec/acceptance/nodesets/debian-8-x86_64-openstack.yml new file mode 100644 index 000000000..003b6f4bd --- /dev/null +++ b/kmod/spec/acceptance/nodesets/debian-8-x86_64-openstack.yml @@ -0,0 +1,14 @@ +HOSTS: + debian-8-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-8-amd64 + hypervisor : openstack + flavor: m1.small + image: debian-8-latest + user: debian +CONFIG: + type: foss + log_level: debug + openstack_network: default diff --git a/kmod/spec/acceptance/nodesets/debian-8-x86_64-vagrant.yml b/kmod/spec/acceptance/nodesets/debian-8-x86_64-vagrant.yml new file mode 100644 index 000000000..5cc7f0c5a --- /dev/null +++ b/kmod/spec/acceptance/nodesets/debian-8-x86_64-vagrant.yml @@ -0,0 +1,11 @@ +HOSTS: + debian-8-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-8-amd64 + hypervisor : vagrant + box : camptocamp/debian-8-amd64 +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/ubuntu-10.04-x86_64-docker.yml b/kmod/spec/acceptance/nodesets/ubuntu-10.04-x86_64-docker.yml new file mode 100644 index 000000000..933dee605 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/ubuntu-10.04-x86_64-docker.yml @@ -0,0 +1,13 @@ +HOSTS: + ubuntu-1004-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-10.04-amd64 + hypervisor : docker + image: ubuntu:10.04 + # This stops the image from being deleted on completion, speeding up the process. + docker_preserve_image: true +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/ubuntu-12.04-x86_64-docker.yml b/kmod/spec/acceptance/nodesets/ubuntu-12.04-x86_64-docker.yml new file mode 100644 index 000000000..f0ec72b86 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/ubuntu-12.04-x86_64-docker.yml @@ -0,0 +1,15 @@ +HOSTS: + ubuntu-1204-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-12.04-amd64 + hypervisor : docker + image: ubuntu:12.04 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y wget' +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/ubuntu-12.04-x86_64-openstack.yml b/kmod/spec/acceptance/nodesets/ubuntu-12.04-x86_64-openstack.yml new file mode 100644 index 000000000..f81b04b74 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/ubuntu-12.04-x86_64-openstack.yml @@ -0,0 +1,14 @@ +HOSTS: + ubuntu-1204-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-12.04-amd64 + hypervisor : openstack + flavor: m1.small + image: ubuntu-1204-latest + user: ubuntu +CONFIG: + type: foss + log_level: debug + openstack_network: default diff --git a/kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-docker.yml b/kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-docker.yml new file mode 100644 index 000000000..6fb9281e5 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-docker.yml @@ -0,0 +1,15 @@ +HOSTS: + ubuntu-1404-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-14.04-amd64 + hypervisor : docker + image: ubuntu:14.04 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y wget' +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-openstack.yml b/kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-openstack.yml new file mode 100644 index 000000000..2eeb912d9 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-openstack.yml @@ -0,0 +1,14 @@ +HOSTS: + ubuntu-1404-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-14.04-amd64 + hypervisor : openstack + flavor: m1.small + image: ubuntu-1404-latest + user: ubuntu +CONFIG: + type: foss + log_level: debug + openstack_network: default diff --git a/kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-vagrant.yml b/kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-vagrant.yml new file mode 100644 index 000000000..3b3769539 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/ubuntu-14.04-x86_64-vagrant.yml @@ -0,0 +1,11 @@ +HOSTS: + ubuntu-1404-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-14.04-amd64 + hypervisor : vagrant + box : puppetlabs/ubuntu-14.04-64-nocm +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/ubuntu-14.10-x86_64-docker.yml b/kmod/spec/acceptance/nodesets/ubuntu-14.10-x86_64-docker.yml new file mode 100644 index 000000000..2be425c54 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/ubuntu-14.10-x86_64-docker.yml @@ -0,0 +1,15 @@ +HOSTS: + ubuntu-1410-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-14.10-amd64 + hypervisor : docker + image: ubuntu:14.10 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y wget' +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/ubuntu-14.10-x86_64-openstack.yml b/kmod/spec/acceptance/nodesets/ubuntu-14.10-x86_64-openstack.yml new file mode 100644 index 000000000..58a2acd26 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/ubuntu-14.10-x86_64-openstack.yml @@ -0,0 +1,14 @@ +HOSTS: + ubuntu-1410-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-14.10-amd64 + hypervisor : openstack + flavor: m1.small + image: ubuntu-1410-latest + user: ubuntu +CONFIG: + type: foss + log_level: debug + openstack_network: default diff --git a/kmod/spec/acceptance/nodesets/ubuntu-15.04-x86_64-docker.yml b/kmod/spec/acceptance/nodesets/ubuntu-15.04-x86_64-docker.yml new file mode 100644 index 000000000..caed722c2 --- /dev/null +++ b/kmod/spec/acceptance/nodesets/ubuntu-15.04-x86_64-docker.yml @@ -0,0 +1,15 @@ +HOSTS: + ubuntu-1504-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-15.04-amd64 + hypervisor : docker + image: ubuntu:15.04 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y wget' +CONFIG: + type: foss + log_level: debug diff --git a/kmod/spec/acceptance/nodesets/ubuntu-15.04-x86_64-openstack.yml b/kmod/spec/acceptance/nodesets/ubuntu-15.04-x86_64-openstack.yml new file mode 100644 index 000000000..22ef76c4e --- /dev/null +++ b/kmod/spec/acceptance/nodesets/ubuntu-15.04-x86_64-openstack.yml @@ -0,0 +1,14 @@ +HOSTS: + ubuntu-1504-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-15.04-amd64 + hypervisor : openstack + flavor: m1.small + image: ubuntu-1504-latest + user: ubuntu +CONFIG: + type: foss + log_level: debug + openstack_network: default diff --git a/kmod/spec/classes/kmod_spec.rb b/kmod/spec/classes/kmod_spec.rb new file mode 100644 index 000000000..f37f8c096 --- /dev/null +++ b/kmod/spec/classes/kmod_spec.rb @@ -0,0 +1,26 @@ +require 'spec_helper' + +describe 'kmod', :type => :class do + + on_supported_os.each do |os, facts| + context "on #{os} with augeas 0.8.9" do + let(:facts) do facts.merge({:augeasversion => '0.8.9'}) end + it do + expect { + should compile + }.to raise_error(/Augeas 0.10.0 or higher required/) + end + end + context "on #{os}" do + let(:facts) do + facts.merge( { :augeasversion => '1.2.0' } ) + end + + it { should contain_class('kmod') } + it { should contain_file('/etc/modprobe.d').with({ 'ensure' => 'directory' }) } + ['modprobe.conf','aliases.conf','blacklist.conf'].each do |file| + it { should contain_file("/etc/modprobe.d/#{file}").with({ 'ensure' => 'file' }) } + end + end + end +end diff --git a/kmod/spec/defines/kmod_alias_spec.rb b/kmod/spec/defines/kmod_alias_spec.rb new file mode 100644 index 000000000..298009e40 --- /dev/null +++ b/kmod/spec/defines/kmod_alias_spec.rb @@ -0,0 +1,42 @@ +require 'spec_helper' + +describe 'kmod::alias', :type => :define do + let(:title) { 'foo' } + + on_supported_os.each do |os, facts| + context "on #{os}" do + let(:facts) do + facts.merge( {:augeasversion => '1.2.0'} ) + end + + let(:default_params) do { :source =>'bar', :file => '/baz' } end + + context 'when a file is specified' do + let(:params) do default_params end + it { should contain_kmod__alias('foo') } + it { should contain_kmod__setting('kmod::alias foo') .with({ + 'ensure' => 'present', + 'module' => 'foo', + 'file' => '/baz', + 'category' => 'alias', + 'option' => 'modulename', + 'value' => 'bar' + }) } + end + + context 'when a file is specified and an aliasname' do + let(:params) do default_params.merge!({ :aliasname => 'tango' }) end + it { should contain_kmod__alias('foo') } + it { should contain_kmod__setting('kmod::alias foo') .with({ + 'ensure' => 'present', + 'module' => 'tango', + 'file' => '/baz', + 'category' => 'alias', + 'option' => 'modulename', + 'value' => 'bar' + }) } + end + + end + end +end diff --git a/kmod/spec/defines/kmod_blacklist_spec.rb b/kmod/spec/defines/kmod_blacklist_spec.rb new file mode 100644 index 000000000..908eebd65 --- /dev/null +++ b/kmod/spec/defines/kmod_blacklist_spec.rb @@ -0,0 +1,49 @@ +require 'spec_helper' + +describe 'kmod::blacklist', :type => :define do + let(:title) { 'foo' } + + on_supported_os.each do |os, facts| + context "on #{os}" do + let(:facts) do + facts.merge({ + :augeasversion => '1.2.0', + }) + end + + context 'when ensure is set to present' do + let(:params) do { :ensure => 'present', :file => '/bar/baz' } end + it { should contain_kmod__blacklist('foo') } + it { should contain_kmod__setting('kmod::blacklist foo').with({ + 'ensure' => 'present', + 'category' => 'blacklist', + 'module' => 'foo', + 'file' => '/bar/baz' + }) } + end + + + context 'when file is not specified' do + let(:params) do { :ensure => 'present' } end + it { should contain_kmod__blacklist('foo') } + it { should contain_kmod__setting('kmod::blacklist foo').with({ + 'ensure' => 'present', + 'category' => 'blacklist', + 'module' => 'foo', + 'file' => '/etc/modprobe.d/blacklist.conf' + }) } + end + + context 'when ensure is set to absent' do + let(:params) do { :ensure => 'absent', :file => '/bar/baz' } end + it { should contain_kmod__blacklist('foo') } + it { should contain_kmod__setting('kmod::blacklist foo').with({ + 'ensure' => 'absent', + 'category' => 'blacklist', + 'module' => 'foo', + 'file' => '/bar/baz' + }) } + end + end + end +end diff --git a/kmod/spec/defines/kmod_install_spec.rb b/kmod/spec/defines/kmod_install_spec.rb new file mode 100644 index 000000000..a8937ccf9 --- /dev/null +++ b/kmod/spec/defines/kmod_install_spec.rb @@ -0,0 +1,26 @@ +require 'spec_helper' + +describe 'kmod::install', :type => :define do + let(:title) { 'foo' } + + on_supported_os.each do |os, facts| + context "on #{os}" do + let(:facts) do + facts.merge({ + :augeasversion => '1.2.0', + }) + end + + let(:params) do { :ensure => 'present', :command => '/bin/true', :file => '/etc/modprobe.d/modprobe.conf' } end + it { should contain_kmod__install('foo') } + it { should contain_kmod__setting('kmod::install foo').with({ + 'ensure' => 'present', + 'category' => 'install', + 'module' => 'foo', + 'option' => 'command', + 'value' => '/bin/true', + 'file' => '/etc/modprobe.d/modprobe.conf' + }) } + end + end +end diff --git a/kmod/spec/defines/kmod_load_spec.rb b/kmod/spec/defines/kmod_load_spec.rb new file mode 100644 index 000000000..0e124b23e --- /dev/null +++ b/kmod/spec/defines/kmod_load_spec.rb @@ -0,0 +1,67 @@ +require 'spec_helper' + +describe 'kmod::load', :type => :define do + let(:title) { 'foo' } + + on_supported_os.each do |os, facts| + context "on #{os}" do + let(:facts) do + facts.merge( { :augeasversion => '1.2.0' } ) + end + + context 'with ensure set to present' do + let(:params) do { :ensure => 'present', :file => '/foo/bar' } end + it { should contain_kmod__load('foo') } + it { should contain_exec('modprobe foo').with({'unless' => "egrep -q '^foo ' /proc/modules"}) } + + case facts[:osfamily] + when 'Debian' + it { should contain_augeas('Manage foo in /foo/bar').with({ + 'incl' => '/foo/bar', + 'lens' => 'Modules.lns', + 'changes' => "clear 'foo'" + }) } + when 'Suse' + it { should contain_augeas('sysconfig_kernel_MODULES_LOADED_ON_BOOT_foo').with({ + 'incl' => '/foo/bar', + 'lens' => 'Shellvars_list.lns', + 'changes' => "set MODULES_LOADED_ON_BOOT/value[.='foo'] 'foo'" + }) } + when 'RedHat' + it { should contain_file('/etc/sysconfig/modules/foo.modules').with({ + 'ensure' => 'present', + 'mode' => '0755', + 'content' => /exec \/sbin\/modprobe foo > \/dev\/null 2>&1/ + })} + end + end + + context 'with ensure set to absent' do + let(:params) do { :ensure => 'absent', :file => '/foo/bar' } end + it { should contain_kmod__load('foo') } + it { should contain_exec('modprobe -r foo').with({ 'onlyif' => "egrep -q '^foo ' /proc/modules" }) } + + case facts[:osfamily] + when 'Debian' + it { should contain_augeas('Manage foo in /foo/bar').with({ + 'incl' => '/foo/bar', + 'lens' => 'Modules.lns', + 'changes' => "rm 'foo'" + })} + when 'Suse' + it { should contain_augeas('sysconfig_kernel_MODULES_LOADED_ON_BOOT_foo').with({ + 'incl' => '/foo/bar', + 'lens' => 'Shellvars_list.lns', + 'changes' => "rm MODULES_LOADED_ON_BOOT/value[.='foo']" + }) } + when 'RedHat' + it { should contain_file('/etc/sysconfig/modules/foo.modules').with({ + 'ensure' => 'absent', + 'mode' => '0755', + 'content' => /exec \/sbin\/modprobe foo > \/dev\/null 2>&1/ + })} + end + end + end + end +end diff --git a/kmod/spec/defines/kmod_setting_spec.rb b/kmod/spec/defines/kmod_setting_spec.rb new file mode 100644 index 000000000..5f3673427 --- /dev/null +++ b/kmod/spec/defines/kmod_setting_spec.rb @@ -0,0 +1,40 @@ +require 'spec_helper' + +describe 'kmod::setting', :type => :define do + let(:title) { 'foo' } + + on_supported_os.each do |os, facts| + context "on #{os}" do + let(:facts) do + facts.merge({ + :augeasversion => '1.2.0', + }) + end + + let(:default_params) do { :file => 'modprobe.conf' } end + let(:params) do default_params end + + context 'add an alias' do + let(:params) do default_params.merge({ :category => 'alias', :option => 'modulename', :value => 'tango' }) end + it { should contain_kmod__setting('foo')} + it { should contain_augeas('kmod::setting foo foo').with({ + 'incl' => 'modprobe.conf', + 'lens' => 'Modprobe.lns', + 'changes' => [ "set alias[. = 'foo'] foo", "set alias[. = 'foo']/modulename tango" ], + 'require' => 'File[modprobe.conf]' + })} + end + context 'add a blacklist' do + let(:params) do { :file => '/etc/modprobe.d/blacklist.conf', :category => 'blacklist' } end + it { should contain_kmod__setting('foo')} + it { should contain_augeas('kmod::setting foo foo').with({ + 'incl' => '/etc/modprobe.d/blacklist.conf', + 'lens' => 'Modprobe.lns', + 'changes' => [ "set blacklist[. = 'foo'] foo" ], + 'require' => 'File[/etc/modprobe.d/blacklist.conf]' + })} + end + + end + end +end diff --git a/kmod/spec/spec.opts b/kmod/spec/spec.opts new file mode 100644 index 000000000..91cd6427e --- /dev/null +++ b/kmod/spec/spec.opts @@ -0,0 +1,6 @@ +--format +s +--colour +--loadby +mtime +--backtrace diff --git a/kmod/spec/spec_helper.rb b/kmod/spec/spec_helper.rb new file mode 100644 index 000000000..94d30d5ce --- /dev/null +++ b/kmod/spec/spec_helper.rb @@ -0,0 +1,42 @@ +require 'puppetlabs_spec_helper/module_spec_helper' +require 'rspec-puppet-facts' +include RspecPuppetFacts + + +RSpec.configure do |c| + c.include PuppetlabsSpec::Files + + c.before :each do + # Store any environment variables away to be restored later + @old_env = {} + ENV.each_key {|k| @old_env[k] = ENV[k]} + + c.strict_variables = Gem::Version.new(Puppet.version) >= Gem::Version.new('3.5') + Puppet.features.stubs(:root?).returns(true) + end + + c.after :each do + PuppetlabsSpec::Files.cleanup + end +end + +require 'pathname' +dir = Pathname.new(__FILE__).parent +Puppet[:modulepath] = File.join(dir, 'fixtures', 'modules') + +# There's no real need to make this version dependent, but it helps find +# regressions in Puppet +# +# 1. Workaround for issue #16277 where default settings aren't initialised from +# a spec and so the libdir is never initialised (3.0.x) +# 2. Workaround for 2.7.20 that now only loads types for the current node +# environment (#13858) so Puppet[:modulepath] seems to get ignored +# 3. Workaround for 3.5 where context hasn't been configured yet, +# ticket https://tickets.puppetlabs.com/browse/MODULES-823 +# +ver = Gem::Version.new(Puppet.version.split('-').first) +if Gem::Requirement.new("~> 2.7.20") =~ ver || Gem::Requirement.new("~> 3.0.0") =~ ver || Gem::Requirement.new("~> 3.5") =~ ver || Gem::Requirement.new("~> 4.0") + puts "augeasproviders: setting Puppet[:libdir] to work around broken type autoloading" + # libdir is only a single dir, so it can only workaround loading of one external module + Puppet[:libdir] = "#{Puppet[:modulepath]}/augeasproviders_core/lib" +end diff --git a/kmod/templates/redhat.modprobe.erb b/kmod/templates/redhat.modprobe.erb new file mode 100644 index 000000000..389df3418 --- /dev/null +++ b/kmod/templates/redhat.modprobe.erb @@ -0,0 +1,5 @@ +#!/bin/sh + +# file managed by puppet + +exec /sbin/modprobe <%= @name %> > /dev/null 2>&1 From 76cf6f3ef47ba118b70d7aef5e37ceede3f02e9d Mon Sep 17 00:00:00 2001 From: Lukas Bezdicka Date: Thu, 4 Feb 2016 15:44:44 +0100 Subject: [PATCH 24/35] Update apache to 7e57138aa9a27ffbd07069243efffaa39a7a865c 7e57138aa9a27ffbd07069243efffaa39a7a865c Merge pull request #1355 from BigAl/MODULES-3017 c1e70c9e589e600fb4914ab21a8d82f6c3e88010 Merge pull request #1338 from timogoebel/feature_more_file_mode 5a1fdd32b14b630a98fe9d48d128c463d1b437e3 Merge pull request #1347 from jyaworski/add_apache_version_fact 595a55ccaaf64c603b4f951059fae59d4eca0797 Merge pull request #1356 from hunner/fix_pagespeed_spec fe1dae9f59c756b86d05dd4e3e921b2838031e6d Need to know where mod_dir is b1f60c7c4297de78350a10cbede8ed1149df0fdd allow status code on redirect match to be optional and not a requirement as per apache documentation http://httpd.apache.org/docs/2.2/mod/mod_alias.html#redirectmatch c1ba7e6b813d072f54cc45a7e28bc8f3e7bc1746 Merge pull request #1354 from hunner/fix_passenger 9996cb23a118ee14806197a8597be1c0cbe2e70b Fix passenger on redhat systems 5150b35c48d888c95504bbc91242163354a0f8e2 Merge pull request #1296 from whotwagner/master 4c4ddaa6ef352ed85b592c0913a5331315817e67 Merge pull request #1350 from rvincentatprokarmadotcom/master 2f78dab061378ed6e0e5b7c6aa670dd2625dea14 Bugfix: require concat, not file f077061b72515a9b336604f22ef7af1801fb2dd5 Merge pull request #1352 from tphoney/ldap_parameters ed74be3277eac484254e0bb887c736b35c14389e adding docs and tests for ldap parameter additions 9fa1dd71aef812bde52337ea6a4b589d53ca10f1 Added support cache related options to apache::mod::ldap 3d34f69f60ef39555a990efe2ca0881d8b3dfc7a Merge pull request #1351 from tphoney/threadlimit-order f0a9223d6c906f23da217099e20515c13d9ccb31 Merge pull request #1322 from BigAl/bug/MODULES-2958 a91d9d7293c0cb26b9bf7b1344ced6fb3dfadcd4 ThreadLimit needs to be above MaxClients or it is ignored. https://bz.apache.org/bugzilla/show_bug.cgi?id=46113 5e431279017e900966fc843d75a97c943448d016 Add apache_version fact 081bdc449f992114598bb0ca3f7a331f1f5cd114 Merge pull request #1344 from fbarbeira/patch-1 44f5231c184934e61e56854438e9413b354b2ac8 Merge pull request #1343 from DavidS/fm-4049-update-msync c748b4502da5534e0ffafd24e5b688e2c1585cd8 (FM-4049) Update to current msync configs [2c99161] 5532bd08e9d71f56c65701c8134a726db48751a6 Remove white space. 95694e6ef16f899caa308bb0c7e4a6cd65e775c0 Merge pull request #1337 from derdanne/gentoo-fix-modules fc3bea1dace807b786c437edcd5311c8d3a3a19a add file_mode to mod manifests 2c6a22a19777990d7e55e7ae7c70c10273804b16 Merge pull request #1333 from timogoebel/filemode 2c67206b2a1b212dff468dfc54ba49c76cd545a2 Merge pull request #1335 from timogoebel/fix_wildcard_ip f2fc48e15159ea30626c4816cb5233b28a6066c5 Merge pull request #1336 from tphoney/fm-3801 6d4fcd8160d315a02a4e1fe441e550710f9b7e40 work around pup_5016, debian8 2015.2 service fault dbfc3844541b745cef87251a32db94ab6178bd1a update module handling for gentoo when used apache 2.4 add module authnz_external as external package in gentoo 7b17dad57ee5cc2aec51d553950cc093eefcaf06 fix vhosts listen to wildcard ip 0f01b59c8ccb303b78ac1cd433d6144f844d6daf add paramter to set config file permissions 5a277163de3807235f7f2fb4b11ec3d1f0b2d05f Merge pull request #1332 from tphoney/fm-3801 ee47dcbecdf93fa5e1b867495f3bbaf65bd3b84a install epel for RH in acceptance. clean itk test 6f14e7d2a6c84a9c8b95a6be24f492faff745ef6 Merge pull request #1330 from tphoney/fm-3801 80fb727f1a524261dcbf3fbbf2f3d8af6209066f fixing apache_parameters_spec.rb 050fee30b58e248d20628b877217a45cccc0b056 Merge pull request #1329 from tphoney/fm-3801 42bf228282fb540eb084d048f1d69748dc48f87d modify tests for debian 8 and add tests 5442a987d6bc5fed07ada84d226d9989452b4b92 Merge pull request #1327 from tphoney/proxy_test_fix 4a8211097975c1680eb575963bd27406680e6f9d a path is needed for ProxyPassReverse ff8ebaffe9d257579d68176d9d77877634da1e23 Merge pull request #1326 from igalic/proxy-pass-no-location 99add117df452d2c513a3a7c36d9fd0416e696b9 proxy: remove workaround for old broken clients ac31d2ec0fcaf2e962cc3d5b0590b512d69744d8 MODULES-2958 : correct CustomLog syslog entry 9507b9e9b801cc789280683423c244e684327410 Merge pull request #1323 from traylenator/pass_max_queue_size a2c49b0086b798acdbc63c770c949724dfa06604 Merge pull request #1324 from tphoney/redhat_itk_test_fix d4fb48aa501e0e86d4a57efd3e5fade978a24fdb itk tests on redhat need epel 2a979d276bb7e45570fc41b98ad5220cf36f99a0 Merge pull request #1321 from ryno75/master bd6239c59d3f77962ba9785024cf87f2ef7ff74c (MODULES-2964) Enable PassengerMaxRequestQueueSize to be set b1d20042f02931a05c64a29fd47c9078f6c0ab43 Add some mod::passenger documentation. d9034acf0040bb237fdc85eda29575a62d659c6d Fixing error in Amazon $operatingsystem comparison 1f3c7b904f1d2dd398645876e9a1af01a417ce84 Merge pull request #1320 from mpolenchuk/master cf92e835bbdd4066fe0527398911904b3fc76aa0 Update docs for predefined LogFormats 4c078b7dc6b27fa0d10c4bf01a57cb3f7d11e590 Merge pull request #1317 from jlambert121/proxy_pass_options 15fc6569ba32f062378c617dd13c8f2042dd6488 Merge pull request #1319 from tphoney/redhat_mod_security_fix 9dd474b760178fe42c7e568999099c1c87eab2c1 fix ordering of catalogue for redhat 7 ca432ee289cb15a23d76b76662a7fb9f56bdfbbf Merge pull request #1292 from cholyoak/shib2_debian 4ae775a3662f668deaf6a972db48947b0fa3bafb Fix paths and packages for the shib2 module on Debian pre Jessie 8a350af5cf956fe8f346df6786e5b157e8024dda Merge pull request #1316 from MiamiOH/master db5b0bfb09959622ea1c6cc299b58057b94c588f MODULES-2956: Enable options within location block on proxy_match 2e6cd7aad53a6e9db97c618fa0a432fc3182574c Support itk on redhat 2b43798124719b4672392818d60e05134c2088dc Merge pull request #1315 from pxsloot/ticket/2948-proxypass_parameters_ensure_order 19e3ee61e8a1a0ab5d6b0288038b6750ad4c8b7f (#2948) ensure order of ProxyPass and ProxyPassMatch parameters a78617b1919f44ab32fb88219783d836a77db148 Merge pull request #1308 from mpolenchuk/master ca5e7e5dd3b727352da7ca5328305188987d3db7 Merge pull request #1311 from occelebi/ssl 5ab89d9220b0995426db15d6b4156789ba919599 Merge pull request #1313 from jewjitsu/fix_indent bb456f1abf1a6a77d14fd37cbc500c77204e9559 Merge pull request #1310 from occelebi/proxy 12b695ab82c9a344e87d3f73b4dae1864cb90dac the custom fragment needs to be indented two spaces so that it lines up with the rest of the configuration a38918431c0e3d41cd8f6031fab564398215d477 Merge pull request #1309 from occelebi/domain 2a6085929cba0ca4be759ea4d514143dced11dac Merge pull request #1305 from jasonhancock/bug-el7-module-dir e825422b0080e66b32cd05ed51b5ccae69325c74 EL7 uses conf.modules.d directory for modules. 0ab0e393f0e7254bc28f69b624cf212f120a254c Support the mod_ssl SSLProxyVerify directive 7db484aacbebba67a879b195d83bfc604c619daf Add ProxyPreserveHost off mode explicitly f001dfb231f41aac73d97a7878eb29c9e4880c24 Support ProxPassReverseCookieDomain directive (mod_proxy) 05437c0b6e4dd084b2336565aa13a8d152f97071 Merge pull request #1306 from quixoten/request_headers_order 6b8aa3ae30d5033b88d57eae622a8bfef91f1bf8 Put headers and request headers before proxy cf84e97e6a75dfaa9195d98b0917bfd2168badd8 Merge pull request #1304 from roidelapluie/vhost-directory-proxy-provider 1dfc02d9036e501570aebabc8d02945073e33a6d Added support for modsecurity parameter SecPcreMatchLimit and SecPcreMatchLimitRecursion 5ed3dbe93bda0b8c066ee6f712b6b78a9ef0810f Merge pull request #1301 from timogoebel/fix_rewrite_validation e6837d8a5e8f481295fe1a45d9d1a6bd950c4028 Add X-Forwarded-For into log_formats defaults 57d18900ae86c27f224c3e72b69b81ac17c8a79d Merge pull request #1307 from bmjen/fix-fastcgi e90496f9671333eec2ffb1eeb1afe7bbdfc27059 (maint) fixes fastcgi tests on ubuntu lucid 407af7b70d90ef2897d062bf00ca11f78cd25608 Support proxy provider for vhost directories 704b3e1b401384515c40630d0248d65455493c75 Merge pull request #1303 from bmjen/fix-fastcgi 2e15018dd24d74c78a50a9483cf39b791f3ff7ec (maint) fixes the fastcgi test for debian and older ubuntu releases 791c20f33518245c691739aa06880cbe5cfb832b Merge pull request #1302 from pabelanger/temp/testing aae1a6664efc200640eccd203db6831dd57f8d71 Limit fastcgi testing to Debian osfamily 963597dd55f95e47a31597b4df0db83ea5b21186 Merge pull request #1300 from pabelanger/temp/pcci-centos 530adf44da90a5d7438f13fa797afddc0da60ac6 Skip fastcgi acceptance testing for Centos 7 924511e1b1ceef79cd06a4833f883534dd48c125 fix validation error when empty array is passed as rewrites parameter Change-Id: Icf815761c9f915488d09a85c98de22e1dd1c10f2 --- Puppetfile | 2 +- apache/.gitignore | 1 + apache/.rspec | 2 + apache/.travis.yml | 15 +- apache/CONTRIBUTING.md | 6 +- apache/Gemfile | 22 +- apache/README.md | 593 ++++++++++-------- apache/Rakefile | 3 +- apache/lib/facter/apache_version.rb | 8 + .../puppet/parser/functions/enclose_ipv6.rb | 18 +- apache/manifests/balancer.pp | 2 +- apache/manifests/fastcgi/server.pp | 2 +- apache/manifests/init.pp | 24 +- apache/manifests/mod.pp | 10 +- apache/manifests/mod/alias.pp | 1 + apache/manifests/mod/auth_cas.pp | 1 + apache/manifests/mod/auth_mellon.pp | 1 + apache/manifests/mod/authnz_ldap.pp | 1 + apache/manifests/mod/autoindex.pp | 1 + apache/manifests/mod/cgid.pp | 1 + apache/manifests/mod/dav_fs.pp | 1 + apache/manifests/mod/deflate.pp | 1 + apache/manifests/mod/dir.pp | 1 + apache/manifests/mod/disk_cache.pp | 1 + apache/manifests/mod/event.pp | 3 +- apache/manifests/mod/expires.pp | 1 + apache/manifests/mod/ext_filter.pp | 1 + apache/manifests/mod/fastcgi.pp | 1 + apache/manifests/mod/fcgid.pp | 1 + apache/manifests/mod/geoip.pp | 1 + apache/manifests/mod/info.pp | 1 + apache/manifests/mod/itk.pp | 23 +- apache/manifests/mod/ldap.pp | 6 + apache/manifests/mod/mime.pp | 1 + apache/manifests/mod/mime_magic.pp | 1 + apache/manifests/mod/negotiation.pp | 1 + apache/manifests/mod/nss.pp | 1 + apache/manifests/mod/pagespeed.pp | 1 + apache/manifests/mod/passenger.pp | 63 +- apache/manifests/mod/peruser.pp | 3 +- apache/manifests/mod/php.pp | 2 +- apache/manifests/mod/prefork.pp | 2 +- apache/manifests/mod/proxy.pp | 1 + apache/manifests/mod/proxy_html.pp | 1 + apache/manifests/mod/remoteip.pp | 1 + apache/manifests/mod/rpaf.pp | 1 + apache/manifests/mod/security.pp | 7 +- apache/manifests/mod/setenvif.pp | 1 + apache/manifests/mod/ssl.pp | 1 + apache/manifests/mod/status.pp | 3 +- apache/manifests/mod/suphp.pp | 1 + apache/manifests/mod/userdir.pp | 1 + apache/manifests/mod/worker.pp | 2 +- apache/manifests/mod/wsgi.pp | 1 + apache/manifests/params.pp | 76 ++- apache/manifests/vhost.pp | 84 +-- apache/manifests/vhost/custom.pp | 2 +- .../spec/acceptance/apache_parameters_spec.rb | 18 +- apache/spec/acceptance/apache_ssl_spec.rb | 11 +- apache/spec/acceptance/class_spec.rb | 107 ++-- apache/spec/acceptance/default_mods_spec.rb | 59 +- apache/spec/acceptance/itk_spec.rb | 56 +- apache/spec/acceptance/mod_dav_svn_spec.rb | 27 +- apache/spec/acceptance/mod_deflate_spec.rb | 26 +- apache/spec/acceptance/mod_mime_spec.rb | 26 +- .../spec/acceptance/mod_negotiation_spec.rb | 50 +- apache/spec/acceptance/mod_pagespeed_spec.rb | 30 +- apache/spec/acceptance/mod_passenger_spec.rb | 71 ++- apache/spec/acceptance/mod_php_spec.rb | 44 +- apache/spec/acceptance/mod_proxy_html_spec.rb | 20 +- apache/spec/acceptance/mod_security_spec.rb | 54 +- apache/spec/acceptance/mod_suphp_spec.rb | 6 +- .../acceptance/nodesets/centos-59-x64.yml | 10 + .../acceptance/nodesets/centos-64-x64-pe.yml | 12 + .../acceptance/nodesets/centos-65-x64.yml | 10 + .../acceptance/nodesets/debian-82-x64.yml | 10 + apache/spec/acceptance/prefork_worker_spec.rb | 57 +- apache/spec/acceptance/service_spec.rb | 21 +- apache/spec/acceptance/version.rb | 3 +- apache/spec/acceptance/vhost_spec.rb | 150 ++++- apache/spec/classes/apache_spec.rb | 12 +- apache/spec/classes/mod/itk_spec.rb | 49 ++ apache/spec/classes/mod/ldap_spec.rb | 14 +- apache/spec/classes/mod/passenger_spec.rb | 7 + apache/spec/classes/mod/php_spec.rb | 10 +- apache/spec/classes/mod/security_spec.rb | 2 +- apache/spec/defines/mod_spec.rb | 14 + apache/spec/defines/vhost_spec.rb | 90 ++- apache/spec/spec_helper_acceptance.rb | 22 + apache/spec/unit/apache_version_spec.rb | 20 + .../parser/functions/enclose_ipv6_spec.rb | 8 +- apache/templates/httpd.conf.erb | 3 + apache/templates/mod/ldap.conf.erb | 15 + apache/templates/mod/passenger.conf.erb | 3 + apache/templates/mod/security.conf.erb | 4 +- apache/templates/mod/worker.conf.erb | 2 +- apache/templates/vhost/_access_log.erb | 2 +- apache/templates/vhost/_custom_fragment.erb | 2 +- apache/templates/vhost/_directories.erb | 2 +- apache/templates/vhost/_proxy.erb | 48 +- apache/templates/vhost/_redirect.erb | 10 + apache/templates/vhost/_sslproxy.erb | 3 + 102 files changed, 1388 insertions(+), 846 deletions(-) create mode 100644 apache/.rspec create mode 100644 apache/lib/facter/apache_version.rb create mode 100644 apache/spec/acceptance/nodesets/centos-59-x64.yml create mode 100644 apache/spec/acceptance/nodesets/centos-64-x64-pe.yml create mode 100644 apache/spec/acceptance/nodesets/centos-65-x64.yml create mode 100644 apache/spec/acceptance/nodesets/debian-82-x64.yml create mode 100644 apache/spec/unit/apache_version_spec.rb diff --git a/Puppetfile b/Puppetfile index 39eb6f6ab..156bd3672 100644 --- a/Puppetfile +++ b/Puppetfile @@ -3,7 +3,7 @@ mod 'aodh', :git => 'https://github.com/openstack/puppet-aodh.git' mod 'apache', - :commit => 'cdd81dbf3463c73e6abfaf937f929821baff73dd', + :commit => '7e57138aa9a27ffbd07069243efffaa39a7a865c', :git => 'https://github.com/puppetlabs/puppetlabs-apache.git' mod 'cassandra', diff --git a/apache/.gitignore b/apache/.gitignore index b5db85e05..319027749 100644 --- a/apache/.gitignore +++ b/apache/.gitignore @@ -5,5 +5,6 @@ spec/fixtures/ .vagrant/ .bundle/ coverage/ +log/ .idea/ *.iml diff --git a/apache/.rspec b/apache/.rspec new file mode 100644 index 000000000..16f9cdb01 --- /dev/null +++ b/apache/.rspec @@ -0,0 +1,2 @@ +--color +--format documentation diff --git a/apache/.travis.yml b/apache/.travis.yml index c418ab5f2..e6314a470 100644 --- a/apache/.travis.yml +++ b/apache/.travis.yml @@ -1,18 +1,19 @@ --- sudo: false language: ruby +cache: bundler bundler_args: --without system_tests -script: "bundle exec rake validate && bundle exec rake lint && bundle exec rake spec SPEC_OPTS='--format documentation'" +script: "bundle exec rake validate lint spec" matrix: fast_finish: true include: - - rvm: 1.9.3 - env: PUPPET_GEM_VERSION="~> 3.0" - - rvm: 2.1.5 - env: PUPPET_GEM_VERSION="~> 3.0" - - rvm: 2.1.5 - env: PUPPET_GEM_VERSION="~> 3.0" FUTURE_PARSER="yes" - rvm: 2.1.6 env: PUPPET_GEM_VERSION="~> 4.0" STRICT_VARIABLES="yes" + - rvm: 2.1.5 + env: PUPPET_GEM_VERSION="~> 3.0" FUTURE_PARSER="yes" + - rvm: 2.1.5 + env: PUPPET_GEM_VERSION="~> 3.0" + - rvm: 1.9.3 + env: PUPPET_GEM_VERSION="~> 3.0" notifications: email: false diff --git a/apache/CONTRIBUTING.md b/apache/CONTRIBUTING.md index f1cbde4bb..bfeaa701c 100644 --- a/apache/CONTRIBUTING.md +++ b/apache/CONTRIBUTING.md @@ -159,7 +159,7 @@ If you already have those gems installed, make sure they are up-to-date: With all dependencies in place and up-to-date we can now run the tests: ```shell -% rake spec +% bundle exec rake spec ``` This will execute all the [rspec tests](http://rspec-puppet.com/) tests @@ -178,8 +178,8 @@ installed on your system. You can run them by issuing the following command ```shell -% rake spec_clean -% rspec spec/acceptance +% bundle exec rake spec_clean +% bundle exec rspec spec/acceptance ``` This will now download a pre-fabricated image configured in the [default node-set](./spec/acceptance/nodesets/default.yml), diff --git a/apache/Gemfile b/apache/Gemfile index bfe64b186..ced190e77 100644 --- a/apache/Gemfile +++ b/apache/Gemfile @@ -1,7 +1,7 @@ source ENV['GEM_SOURCE'] || "https://rubygems.org" def location_for(place, fake_version = nil) - if place =~ /^(git:[^#]*)#(.*)/ + if place =~ /^(git[:@][^#]*)#(.*)/ [fake_version, { :git => $1, :branch => $2, :require => false }].compact elsif place =~ /^file:\/\/(.*)/ ['>= 0', { :path => File.expand_path($1), :require => false }] @@ -11,14 +11,16 @@ def location_for(place, fake_version = nil) end group :development, :unit_tests do - gem 'rspec-core', '3.1.7', :require => false - gem 'puppetlabs_spec_helper', :require => false - gem 'simplecov', :require => false - gem 'puppet_facts', :require => false - gem 'json', :require => false + gem 'json', :require => false + gem 'metadata-json-lint', :require => false + gem 'puppet_facts', :require => false + gem 'puppet-blacksmith', :require => false + gem 'puppetlabs_spec_helper', :require => false + gem 'rspec-puppet', '>= 2.3.2', :require => false + gem 'simplecov', :require => false end - group :system_tests do + gem 'beaker-puppet_install_helper', :require => false if beaker_version = ENV['BEAKER_VERSION'] gem 'beaker', *location_for(beaker_version) end @@ -27,12 +29,10 @@ group :system_tests do else gem 'beaker-rspec', :require => false end - gem 'serverspec', :require => false - gem 'beaker-puppet_install_helper', :require => false + gem 'master_manipulator', :require => false + gem 'serverspec', :require => false end - - if facterversion = ENV['FACTER_GEM_VERSION'] gem 'facter', facterversion, :require => false else diff --git a/apache/README.md b/apache/README.md index 57ba525bf..1ee7ce5ae 100644 --- a/apache/README.md +++ b/apache/README.md @@ -66,7 +66,6 @@ [`apache::version`]: #class-apacheversion [`apache::vhost`]: #define-apachevhost [`apache::vhost::custom`]: #define-apachevhostcustom -[`apache::vhost::WSGIImportScript`]: #wsgiimportscript [Apache HTTP Server]: http://httpd.apache.org [Apache modules]: http://httpd.apache.org/docs/current/mod/ [array]: https://docs.puppetlabs.com/puppet/latest/reference/lang_data_array.html @@ -161,6 +160,7 @@ [`mod_mpm_event`]: https://httpd.apache.org/docs/current/mod/event.html [`mod_negotiation`]: http://httpd.apache.org/docs/current/mod/mod_negotiation.html [`mod_pagespeed`]: https://developers.google.com/speed/pagespeed/module/?hl=en +[`mod_passenger`]: https://www.phusionpassenger.com/library/config/apache/reference/ [`mod_php`]: http://php.net/manual/en/book.apache.php [`mod_proxy`]: https://httpd.apache.org/docs/current/mod/mod_proxy.html [`mod_proxy_balancer`]: http://httpd.apache.org/docs/current/mod/mod_proxy_balancer.html @@ -289,19 +289,19 @@ To temporarily disable full Puppet management, set the [`purge_configs`][] param To have Puppet install Apache with the default parameters, declare the [`apache`][] class: -~~~ puppet +``` puppet class { 'apache': } -~~~ +``` The Puppet module applies a default configuration based on your operating system; Debian, Red Hat, FreeBSD, and Gentoo systems each have unique default configurations. These defaults work in testing environments but are not suggested for production, and Puppet recommends customizing the class's parameters to suit your site. Use the [Reference](#reference) section to find information about the class's parameters and their default values. You can customize parameters when declaring the `apache` class. For instance, this declaration installs Apache without the apache module's [default virtual host configuration][Configuring virtual hosts], allowing you to customize all Apache virtual hosts: -~~~ puppet +``` puppet class { 'apache': default_vhost => false, } -~~~ +``` ## Usage @@ -313,41 +313,41 @@ The default [`apache`][] class sets up a virtual host on port 80, listening on a To configure basic [name-based virtual hosts][], specify the [`port`][] and [`docroot`][] parameters in the [`apache::vhost`][] define: -~~~ puppet +``` puppet apache::vhost { 'vhost.example.com': port => '80', docroot => '/var/www/vhost', } -~~~ +``` **Note**: Apache processes virtual hosts in alphabetical order, and server administrators can prioritize Apache's virtual host processing by prefixing a virtual host's configuration file name with a number. The [`apache::vhost`][] define applies a default [`priority`][] of 15, which Puppet interprets by prefixing the virtual host's file name with `15-`. This all means that if multiple sites have the same priority, or if you disable priority numbers by setting the `priority` parameter's value to 'false', Apache still processes virtual hosts in alphabetical order. To configure user and group ownership for `docroot`, use the [`docroot_owner`][] and [`docroot_group`][] parameters: -~~~ puppet +``` puppet apache::vhost { 'user.example.com': port => '80', docroot => '/var/www/user', docroot_owner => 'www-data', docroot_group => 'www-data', } -~~~ +``` #### Configuring virtual hosts with SSL To configure a virtual host to use [SSL encryption][] and default SSL certificates, set the [`ssl`][] parameter. You must also specify the [`port`][] parameter, typically with a value of '443', to accommodate HTTPS requests: -~~~ puppet +``` puppet apache::vhost { 'ssl.example.com': port => '443', docroot => '/var/www/ssl', ssl => true, } -~~~ +``` To configure a virtual host to use SSL and specific SSL certificates, use the paths to the certificate and key in the [`ssl_cert`][] and [`ssl_key`][] parameters, respectively: -~~~ puppet +``` puppet apache::vhost { 'cert.example.com': port => '443', docroot => '/var/www/cert', @@ -355,11 +355,11 @@ apache::vhost { 'cert.example.com': ssl_cert => '/etc/ssl/fourth.example.com.cert', ssl_key => '/etc/ssl/fourth.example.com.key', } -~~~ +``` To configure a mix of SSL and unencrypted virtual hosts at the same domain, declare them with separate [`apache::vhost`] defines: -~~~ puppet +``` puppet # The non-ssl virtual host apache::vhost { 'mix.example.com non-ssl': servername => 'mix.example.com', @@ -374,11 +374,11 @@ apache::vhost { 'mix.example.com ssl': docroot => '/var/www/mix', ssl => true, } -~~~ +``` To configure a virtual host to redirect unencrypted connections to SSL, declare them with separate [`apache::vhost`] defines and redirect unencrypted requests to the virtual host with SSL enabled: -~~~ puppet +``` puppet apache::vhost { 'redirect.example.com non-ssl': servername => 'redirect.example.com', port => '80', @@ -393,33 +393,33 @@ apache::vhost { 'redirect.example.com ssl': docroot => '/var/www/redirect', ssl => true, } -~~~ +``` #### Configuring virtual host port and address bindings -Virtual hosts listen on all IP addresses ('*') by default. To configure the virtual host to listen on a specific IP address, use the [`ip`][] parameter: +Virtual hosts listen on all IP addresses ('\*') by default. To configure the virtual host to listen on a specific IP address, use the [`ip`][] parameter: -~~~ puppet +``` puppet apache::vhost { 'ip.example.com': ip => '127.0.0.1', port => '80', docroot => '/var/www/ip', } -~~~ +``` It is also possible to configure more than one IP address per vhost by using an array of IP addresses for the [`ip`][] parameter: -~~~ puppet +``` puppet apache::vhost { 'ip.example.com': ip => ['127.0.0.1','169.254.1.1'], port => '80', docroot => '/var/www/ip', } -~~~ +``` To configure a virtual host with [aliased servers][], refer to the aliases using the [`serveraliases`][] parameter: -~~~ puppet +``` puppet apache::vhost { 'aliases.example.com': serveraliases => [ 'aliases.example.org', @@ -428,11 +428,11 @@ apache::vhost { 'aliases.example.com': port => '80', docroot => '/var/www/aliases', } -~~~ +``` To set up a virtual host with a wildcard alias for the subdomain mapped to a same-named directory, such as 'http://example.com.loc' mapped to `/var/www/example.com`, define the wildcard alias using the [`serveraliases`][] parameter and the document root with the [`virtual_docroot`][] parameter: -~~~ puppet +``` puppet apache::vhost { 'subdomain.loc': vhost_name => '*', port => '80', @@ -440,11 +440,11 @@ apache::vhost { 'subdomain.loc': docroot => '/var/www', serveraliases => ['*.loc',], } -~~~ +``` To configure a virtual host with [filter rules][], pass the filter directives as an [array][] using the [`filters`][] parameter: -~~~ puppet +``` puppet apache::vhost { 'subdomain.loc': port => '80', filters => [ @@ -455,13 +455,13 @@ apache::vhost { 'subdomain.loc': ], docroot => '/var/www/html', } -~~~ +``` #### Configuring virtual hosts for apps and processors To set up a virtual host with [suPHP][], use the [`suphp_engine`][] parameter to enable the suPHP engine, [`suphp_addhandler`][] parameter to define a MIME type, [`suphp_configpath`][] to set which path suPHP passes to the PHP interpreter, and the [`directory`][] parameter to configure Directory, File, and Location directive blocks: -~~~ puppet +``` puppet apache::vhost { 'suphp.example.com': port => '80', docroot => '/home/appuser/myphpapp', @@ -477,11 +477,11 @@ apache::vhost { 'suphp.example.com': }, ], } -~~~ +``` You can use a set of parameters to configure a virtual host to use the [Web Server Gateway Interface][] (WSGI) for [Python][] applications: -~~~ puppet +``` puppet apache::vhost { 'wsgi.example.com': port => '80', docroot => '/var/www/pythonapp', @@ -500,53 +500,53 @@ apache::vhost { 'wsgi.example.com': wsgi_process_group => 'wsgi', wsgi_script_aliases => { '/' => '/var/www/demo.wsgi' }, } -~~~ +``` Starting in Apache 2.2.16, Apache supports [FallbackResource][], a simple replacement for common RewriteRules. You can set a FallbackResource using the [`fallbackresource`][] parameter: -~~~ puppet +``` puppet apache::vhost { 'wordpress.example.com': port => '80', docroot => '/var/www/wordpress', fallbackresource => '/index.php', } -~~~ +``` **Note**: The `fallbackresource` parameter only supports the 'disabled' value since Apache 2.2.24. To configure a virtual host with a designated directory for [Common Gateway Interface][] (CGI) files, use the [`scriptalias`][] parameter to define the `cgi-bin` path: -~~~ puppet +``` puppet apache::vhost { 'cgi.example.com': port => '80', docroot => '/var/www/cgi', scriptalias => '/usr/lib/cgi-bin', } -~~~ +``` To configure a virtual host for [Rack][], use the [`rack_base_uris`][] parameter: -~~~ puppet +``` puppet apache::vhost { 'rack.example.com': port => '80', docroot => '/var/www/rack', rack_base_uris => ['/rackapp1', '/rackapp2'], } -~~~ +``` #### Configuring IP-based virtual hosts You can configure [IP-based virtual hosts][] to listen on any port and have them respond to requests on specific IP addresses. In this example, we set the server to listen on ports 80 and 81 because the example virtual hosts are _not_ declared with a [`port`][] parameter: -~~~ puppet +``` puppet apache::listen { '80': } apache::listen { '81': } -~~~ +``` Then we configure the IP-based virtual hosts with the [`ip_based`][] parameter: -~~~ puppet +``` puppet apache::vhost { 'first.example.com': ip => '10.0.0.10', docroot => '/var/www/first', @@ -558,11 +558,11 @@ apache::vhost { 'second.example.com': docroot => '/var/www/second', ip_based => true, } -~~~ +``` You can also configure a mix of IP- and [name-based virtual hosts][], and in any combination of [SSL][SSL encryption] and unencrypted configurations. First, we add two IP-based virtual hosts on an IP address (in this example, 10.0.0.10). One uses SSL and the other is unencrypted: -~~~ puppet +``` puppet apache::vhost { 'The first IP-based virtual host, non-ssl': servername => 'first.example.com', ip => '10.0.0.10', @@ -579,11 +579,11 @@ apache::vhost { 'The first IP-based vhost, ssl': docroot => '/var/www/first-ssl', ssl => true, } -~~~ +``` Next, we add two name-based virtual hosts listening on a second IP address (10.0.0.20): -~~~ puppet +``` puppet apache::vhost { 'second.example.com': ip => '10.0.0.20', port => '80', @@ -595,11 +595,11 @@ apache::vhost { 'third.example.com': port => '80', docroot => '/var/www/third', } -~~~ +``` To add name-based virtual hosts that answer on either 10.0.0.10 or 10.0.0.20, you **must** set the [`add_listen`][] parameter to 'false' to disable the default Apache setting of `Listen 80`, as it conflicts with the preceding IP-based virtual hosts. -~~~ puppet +``` puppet apache::vhost { 'fourth.example.com': port => '80', docroot => '/var/www/fourth', @@ -611,7 +611,7 @@ apache::vhost { 'fifth.example.com': docroot => '/var/www/fifth', add_listen => false, } -~~~ +``` ### Installing Apache modules @@ -626,17 +626,17 @@ The Puppet apache module supports installing many common [Apache modules][], oft For example, you can install the `mod_ssl` Apache module with default settings by declaring the [`apache::mod::ssl`][] class: -~~~ puppet +``` puppet class { 'apache::mod::ssl': } -~~~ +``` [`apache::mod::ssl`][] has several parameterized options that you can set when declaring it. For instance, to enable `mod_ssl` with compression enabled, set the [`ssl_compression`][] parameter to 'true': -~~~ puppet +``` puppet class { 'apache::mod::ssl': ssl_compression => true, } -~~~ +``` Note that some modules have prerequisites, which are documented in their references under [`apache::mod::`][]. @@ -644,11 +644,11 @@ Note that some modules have prerequisites, which are documented in their referen You can pass the name of any module that your operating system's package manager can install to the [`apache::mod`][] define to install it. Unlike the specific-module classes, the [`apache::mod`][] define doesn't tailor the installation based on other installed modules or with specific parameters---Puppet only grabs and installs the module's package, leaving detailed configuration up to you. -For example, to install the [`mod_authnz_external`][] Apache module, declare the define with the 'mod_authnz_external' name: +For example, to install the [`mod_authnz_external`][] Apache module, declare the define with the 'mod\_authnz\_external' name: -~~~ puppet +``` puppet apache::mod { 'mod_authnz_external': } -~~~ +``` There's several optional parameters you can specify when defining Apache modules this way. See the [define's reference][`apache::mod`] for details. @@ -656,7 +656,7 @@ There's several optional parameters you can specify when defining Apache modules Add the [`apache::fastcgi::server`][] define to allow [FastCGI][] servers to handle requests for specific files. For example, the following defines a FastCGI server at 127.0.0.1 (localhost) on port 9000 to handle PHP requests: -~~~ puppet +``` puppet apache::fastcgi::server { 'php': host => '127.0.0.1:9000', timeout => 15, @@ -665,17 +665,17 @@ apache::fastcgi::server { 'php': fcgi_alias => '/php.fcgi', file_type => 'application/x-httpd-php' } -~~~ +``` You can then use the [`custom_fragment`] parameter to configure the virtual host to have the FastCGI server handle the specified file type: -~~~ puppet +``` puppet apache::vhost { 'www': ... custom_fragment => 'AddType application/x-httpd-php .php' ... } -~~~ +``` ### Load balancing examples @@ -683,23 +683,23 @@ Apache supports load balancing across groups of servers through the [`mod_proxy` To enable load balancing with [exported resources][], export the [`apache::balancermember`][] define from the load balancer member server: -~~~ puppet +``` puppet @@apache::balancermember { "${::fqdn}-puppet00": balancer_cluster => 'puppet00', url => "ajp://${::fqdn}:8009", options => ['ping=5', 'disablereuse=on', 'retry=5', 'ttl=120'], } -~~~ +``` Then, on the proxy server, create the load balancing group: -~~~ puppet +``` puppet apache::balancer { 'puppet00': } -~~~ +``` To enable load balancing without exporting resources, declare the following on the proxy server: -~~~ puppet +``` puppet apache::balancer { 'puppet00': } apache::balancermember { "${::fqdn}-puppet00": @@ -707,26 +707,26 @@ apache::balancermember { "${::fqdn}-puppet00": url => "ajp://${::fqdn}:8009", options => ['ping=5', 'disablereuse=on', 'retry=5', 'ttl=120'], } -~~~ +``` Then declare the `apache::balancer` and `apache::balancermember` defines on the proxy server. If you need to use the [ProxySet](http://httpd.apache.org/docs/current/mod/mod_proxy.html#proxyset) directive on the balancer, use the [`proxy_set`](#proxy_set) parameter of `apache::balancer`: -~~~ puppet +``` puppet apache::balancer { 'puppet01': proxy_set => { 'stickysession' => 'JSESSIONID', }, } -~~~ +``` ## Reference - [**Public Classes**](#public-classes) - [Class: apache](#class-apache) - [Class: apache::dev](#class-apachedev) - - [Classes: apache::mod::*](#classes-apachemodname) + - [Classes: apache::mod::\*](#classes-apachemodname) - [**Private Classes**](#private-classes) - [Class: apache::confd::no_accf](#class-apacheconfdno_accf) - [Class: apache::default_confd_files](#class-apachedefault_confd_files) @@ -762,15 +762,15 @@ When this class is declared with the default options, Puppet: - Installs the appropriate Apache software package and [required Apache modules](#default_mods) for your operating system. - Places the required configuration files in a directory, with the [default location](#conf_dir) determined by your operating system. -- Configures the server with a default virtual host and standard port ('80') and address ('*') bindings. +- Configures the server with a default virtual host and standard port ('80') and address ('\*') bindings. - Creates a document root directory determined by your operating system, typically `/var/www`. - Starts the Apache service. You can simply declare the default `apache` class: -~~~ puppet +``` puppet class { 'apache': } -~~~ +``` You can establish a default virtual host in this class, by using the [`apache::vhost`][] define, or both. You can also configure additional specific virtual hosts with the [`apache::vhost`][] define. Puppet recommends customizing the `apache` class's declaration with the following parameters, as its default settings are not optimized for production. @@ -884,7 +884,7 @@ Configures a default [SSL][SSL encryption] virtual host. Valid options: Boolean. If 'true', Puppet automatically configures the following virtual host using the [`apache::vhost`][] define: -~~~ puppet +``` puppet apache::vhost { 'default-ssl': port => 443, ssl => true, @@ -893,7 +893,7 @@ apache::vhost { 'default-ssl': serveradmin => $serveradmin, access_log_file => "ssl_${access_log_file}", } -~~~ +``` **Note**: SSL virtual hosts only respond to HTTPS queries. @@ -913,13 +913,13 @@ Configures a specific dev package to use. Valid options: String. Default: 'OS d Example for using httpd 2.4 from the IUS yum repo: -~~~ puppet +``` puppet include ::apache::dev class { 'apache': apache_name => 'httpd24u', dev_packages => 'httpd24u-devel', } -~~~ +``` ##### `docroot` @@ -991,18 +991,19 @@ Changes the error log's verbosity. Valid options: 'alert', 'crit', 'debug', 'eme Define additional [`LogFormat`][] directives. Valid options: A [Hash][], such as: -~~~ puppet +``` puppet $log_formats = { vhost_common => '%v %h %l %u %t \"%r\" %>s %b' } -~~~ +``` There are a number of predefined `LogFormats` in the `httpd.conf` that Puppet creates: -~~~ httpd +``` httpd LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%h %l %u %t \"%r\" %>s %b" common LogFormat "%{Referer}i -> %U" referer LogFormat "%{User-agent}i" agent -~~~ +LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"" forwarded +``` If your `log_formats` parameter contains one of those, it will be overwritten with **your** definition. @@ -1158,6 +1159,10 @@ Controls how Apache handles `TRACE` requests (per [RFC 2616][]) via the [`TraceE Controls whether the systemd module should be installed on Centos 7 servers, this is especially useful if using custom built rpms. This can either be 'true' or 'false, defaults to 'true'. +##### `file_mode` + +The desired permissions mode for config files, in symbolic or numeric notation. This value must be a string. Defaults to '0644'. + ##### `vhost_dir` Changes your virtual host configuration files' location. Default: determined by your operating system. @@ -1169,13 +1174,12 @@ Changes your virtual host configuration files' location. Default: determined by ##### `vhost_include_pattern` -Defines the pattern for files included from the `vhost_dir`. This defaults to '*', also for BC with previous versions of this module. +Defines the pattern for files included from the `vhost_dir`. This defaults to '\*', also for BC with previous versions of this module. However, you may want to set this to a value like '[^.#]\*.conf[^~]' to make sure files accidentally created in this directory (from version control systems, editor backups or the like) are *not* included in your server configuration. -A value of '*.conf' is what is shipped by some operating systems. Also note that this module will, by default, create config files ending -in '.conf'. +A value of '\*.conf' is what is shipped by some operating systems. Also note that this module will, by default, create config files ending in '.conf'. ##### `user` @@ -1219,23 +1223,23 @@ The default value is determined by your operating system: Enables specific [Apache modules][]. You can enable and configure an Apache module by declaring its class. For example, to install and enable [`mod_alias`][] with no icons, you can declare the [`apache::mod::alias`][] class with the `icons_options` parameter set to 'None': -~~~ puppet +``` puppet class { 'apache::mod::alias': icons_options => 'None', } -~~~ +``` The following Apache modules have supported classes, many of which allow for parameterized configuration. You can install other Apache modules with the [`apache::mod`][] define. * `actions` * `alias` (see [`apache::mod::alias`][]) * `auth_basic` -* `auth_cas`* (see [`apache::mod::auth_cas`][]) -* `auth_mellon`* (see [`apache::mod::auth_mellon`][]) +* `auth_cas`\* (see [`apache::mod::auth_cas`][]) +* `auth_mellon`\* (see [`apache::mod::auth_mellon`][]) * `auth_kerb` * `authn_core` * `authn_file` -* `authnz_ldap`* +* `authnz_ldap`\* * `authz_default` * `authz_user` * `autoindex` @@ -1244,10 +1248,10 @@ The following Apache modules have supported classes, many of which allow for par * `cgid` * `dav` * `dav_fs` -* `dav_svn`* -* `deflate` +* `dav_svn`\* +* `deflate\` * `dev` -* `dir`* +* `dir`\* * `disk_cache` (see [`apache::mod::disk_cache`][]) * `event` (see [`apache::mod::event`][]) * `expires` @@ -1258,40 +1262,40 @@ The following Apache modules have supported classes, many of which allow for par * `geoip` (see [`apache::mod::geoip`][]) * `headers` * `include` -* `info`* +* `info`\* * `itk` * `ldap` * `mime` -* `mime_magic`* +* `mime_magic`\* * `negotiation` -* `nss`* +* `nss`\* * `pagespeed` (see [`apache::mod::pagespeed`][]) -* `passenger`* +* `passenger`\* (see [`apache::mod::passenger`][]) * `perl` * `peruser` * `php` (requires [`mpm_module`][] set to `prefork`) -* `prefork`* -* `proxy`* +* `prefork`\* +* `proxy`\* * `proxy_ajp` * `proxy_balancer` -* `proxy_html` +* `proxy_html` (see [`apache::mod::proxy_html`][]) * `proxy_http` * `python` * `reqtimeout` -* `remoteip`* +* `remoteip`\* * `rewrite` -* `rpaf`* +* `rpaf`\* * `setenvif` * `security` -* `shib`* (see [`apache::mod::shib`]) +* `shib`\* (see [`apache::mod::shib`]) * `speling` -* `ssl`* (see [`apache::mod::ssl`][]) -* `status`* (see [`apache::mod::status`][]) +* `ssl`\* (see [`apache::mod::ssl`][]) +* `status`\* (see [`apache::mod::status`][]) * `suphp` -* `userdir`* +* `userdir`\* * `version` * `vhost_alias` -* `worker`* +* `worker`\* * `wsgi` (see [`apache::mod::wsgi`][]) * `xsendfile` @@ -1322,11 +1326,11 @@ Installs and configures [`mod_disk_cache`][] on Apache 2.2, or [`mod_cache_disk` You can specify the cache root by passing a path as a string to the `cache_root` parameter. -~~~ puppet +``` puppet class {'::apache::mod::disk_cache': cache_root => '/path/to/cache', } -~~~ +``` ##### Class: `apache::mod::event` @@ -1349,6 +1353,8 @@ Installs and manages [`mod_auth_cas`][]. Its parameters share names with the Apa The `cas_login_url` and `cas_validate_url` parameters are required; several other parameters have 'undef' default values. +**Note**: The auth\_cas module isn't available on RH/CentOS without providing dependency packages provided by EPEL. See [https://github.com/Jasig/mod_auth_cas]() + **Parameters within `apache::mod::auth_cas`**: - `cas_authoritative`: Determines whether an optional authorization directive is authoritative and binding. Default: 'undef'. @@ -1370,11 +1376,11 @@ The `cas_login_url` and `cas_validate_url` parameters are required; several othe Installs and manages [`mod_auth_mellon`][]. Its parameters share names with the Apache module's directives. -~~~ puppet +``` puppet class{ 'apache::mod::auth_mellon': mellon_cache_size => 101, } -~~~ +``` **Parameters within `apache::mod::auth_mellon`**: @@ -1409,14 +1415,14 @@ Installs [`mod_expires`][] and uses the `expires.conf.erb` template to generate Installs and configures [`mod_ext_filter`][]. -~~~ puppet +``` puppet class { 'apache::mod::ext_filter': ext_filter_define => { 'slowdown' => 'mode=output cmd=/bin/cat preservescontentlength', 'puppetdb-strip' => 'mode=output outtype=application/json cmd="pdb-resource-filter"', }, } -~~~ +``` **Parameters within `apache::mod::ext_filter`**: @@ -1428,7 +1434,7 @@ Installs and configures [`mod_fcgid`][]. The class makes no effort to individually parameterize all available options. Instead, configure `mod_fcgid` using the `options` [hash][]. For example: -~~~ puppet +``` puppet class { 'apache::mod::fcgid': options => { 'FcgidIPCDir' => '/var/run/fcgidsock', @@ -1436,13 +1442,13 @@ class { 'apache::mod::fcgid': 'AddHandler' => 'fcgid-script .fcgi', }, } -~~~ +``` For a full list of options, see the [official `mod_fcgid` documentation][`mod_fcgid`]. If you include `apache::mod::fcgid`, you can set the [`FcgidWrapper`][] per directory, per virtual host. The module must be loaded first; Puppet will not automatically enable it if you set the `fcgiwrapper` parameter in `apache::vhost`. -~~~ puppet +``` puppet include apache::mod::fcgid apache::vhost { 'example.org': @@ -1454,7 +1460,7 @@ apache::vhost { 'example.org': } }, } -~~~ +``` ##### Class: `apache::mod::geoip` @@ -1481,22 +1487,44 @@ Installs and manages [`mod_info`][], which provides a comprehensive overview of - `apache_version`: Default: `$::apache::apache_version`, - `restrict_access`: Determines whether to enable access restrictions. If 'false', the `allow_from` whitelist is ignored and any IP address can access `/server-info`. Valid options: Boolean. Default: 'true'. +##### Class: `apache::mod::passenger` + +Installs and manages [`mod_passenger`][]. + +**Parameters within `apache::mod::passenger`**: +- `passenger_high_performance` Sets the [`PassengerHighPerformance`](https://www.phusionpassenger.com/library/config/apache/reference/#passengerhighperformance). Valid options: on, off. Default: undef. +- `passenger_pool_idle_time` Sets the [`PassengerPoolIdleTime`](https://www.phusionpassenger.com/library/config/apache/reference/#passengerpoolidletime). Default: undef +- `passenger_max_pool_size` Sets the [`PassengerMaxPoolSize`](https://www.phusionpassenger.com/library/config/apache/reference/#passengermaxpoolsize). Default: undef. +- `passenger_max_request_queue_size` Sets the [`PassengerMaxRequestQueueSize`](https://www.phusionpassenger.com/library/config/apache/reference/#passengermaxrequestqueuesize). Default: undef. +- `passenger_max_requests` Sets the [`PassengerMaxRequests`](https://www.phusionpassenger.com/library/config/apache/reference/#passengermaxrequests). Default: undef. +- ... + ##### Class: `apache::mod::ldap` Installs and configures [`mod_ldap`][]. Allows you to modify the [`LDAPTrustedGlobalCert`](https://httpd.apache.org/docs/2.2/mod/mod_ldap.html#ldaptrustedglobalcert) Directive: -~~~puppet +```puppet class { 'apache::mod::ldap': ldap_trusted_global_cert_file => '/etc/pki/tls/certs/ldap-trust.crt' ldap_trusted_global_cert_type => 'CA_DER', + ldap_shared_cache_size => '500000', + ldap_cache_entries => '1024', + ldap_cache_ttl => '600', + ldap_opcache_entries => '1024', + ldap_opcache_ttl => '600', } -~~~ +``` **Parameters within `apache::mod::ldap`:** - `ldap_trusted_global_cert_file`: Path and file name of the trusted CA certificates to use when establishing SSL or TLS connections to an LDAP server. - `ldap_trusted_global_cert_type`: The global trust certificate format. Defaults to 'CA_BASE64'. +- `ldap_shared_cache_size`: Size in bytes of the shared-memory cache. +- `ldap_cache_entries`: Maximum number of entries in the primary LDAP cache. +- `ldap_cache_ttl`: Time that cached items remain valid. +- `ldap_opcache_entries`: Number of entries used to cache LDAP compare operations. +- `ldap_opcache_ttl`: Time that entries in the operation cache remain valid. ##### Class: `apache::mod::negotiation` @@ -1517,7 +1545,7 @@ While this Apache module requires the `mod-pagespeed-stable` package, Puppet **d - `inherit_vhost_config`: Default: 'on'. - `filter_xhtml`: Default: false. -- `cache_path`: Default: '/var/cache/mod_pagespeed/'. +- `cache_path`: Default: '/var/cache/mod\_pagespeed/'. - `log_dir`: Default: '/var/log/pagespeed'. - `memcache_servers`: Default: []. - `rewrite_level`: Default: 'CoreFilters'. @@ -1551,6 +1579,18 @@ While this Apache module requires the `mod-pagespeed-stable` package, Puppet **d The class's parameters correspond to the module's directives. See the [module's documentation][`mod_pagespeed`] for details. +##### Class: `apache::mod::passenger` + +Installs and configures mod\_passenger + +**Parameters within `apache::mod::passenger`**: + +- `manage_repo`: Manage phusionpassenger.com repository. Default: true. + +TODO: The parameters section is incomplete. + +**Note**: The passenger module isn't available on RH/CentOS without providing dependency packages provided by EPEL and mod\_passengers own custom repository. See the `manage_repo` parameter above and [https://www.phusionpassenger.com/library/install/apache/install/oss/el7/]() + ##### Class: `apache::mod::php` Installs and configures [`mod_php`][]. @@ -1567,6 +1607,10 @@ Default values depend on your operating system. - `template`: Defines the path to the `php.conf` template Puppet uses to generate the configuration file. - `content`: Adds arbitrary content to `php.conf`. +##### Class: `apache::mod::proxy_html` + +**Note**: There is no official package available for mod\_proxy\_html and thus it must be made available by means outside of the control of the apache module. + ##### Class: `apache::mod::reqtimeout` Installs and configures [`mod_reqtimeout`][]. @@ -1581,6 +1625,8 @@ Installs the [Shibboleth](http://shibboleth.net/) Apache module `mod_shib`, whic Defining this class enables Shibboleth-specific parameters in `apache::vhost` instances. +**Note**: The shibboleth module isn't available on RH/CentOS without providing dependency packages provided by Shibboleth's repositories. See [http://wiki.aaf.edu.au/tech-info/sp-install-guide]() + ##### Class: `apache::mod::ssl` Installs [Apache SSL features][`mod_ssl`] and uses the `ssl.conf.erb` template to generate its configuration. @@ -1627,10 +1673,12 @@ Installs and configures Trustwave's [`mod_security`][]. It is enabled and runs b - `content_types`: A list of one or more allowed [MIME types][MIME `content-type`]. Default: 'application/x-www-form-urlencoded|multipart/form-data|text/xml|application/xml|application/x-amf' - `crs_package`: Names the package that installs CRS rules. Default: `modsec_crs_package` in [`apache::params`][]. - `modsec_dir`: Defines the path where Puppet installs the modsec configuration and activated rules links. Default: 'On', set by `modsec_dir` in [`apache::params`][]. -${modsec_dir}/activated_rules. +${modsec\_dir}/activated\_rules. - `modsec_secruleengine`: Configures the modsec rules engine. Valid options: 'On', 'Off', and 'DetectionOnly'. Default: `modsec_secruleengine` in [`apache::params`][]. - `restricted_extensions`: A space-separated list of prohibited file extensions. Default: '.asa/ .asax/ .ascx/ .axd/ .backup/ .bak/ .bat/ .cdx/ .cer/ .cfg/ .cmd/ .com/ .config/ .conf/ .cs/ .csproj/ .csr/ .dat/ .db/ .dbf/ .dll/ .dos/ .htr/ .htw/ .ida/ .idc/ .idq/ .inc/ .ini/ .key/ .licx/ .lnk/ .log/ .mdb/ .old/ .pass/ .pdb/ .pol/ .printer/ .pwd/ .resources/ .resx/ .sql/ .sys/ .vb/ .vbs/ .vbproj/ .vsdisco/ .webinfo/ .xsd/ .xsx/'. - `restricted_headers`: A list of restricted headers separated by slashes and spaces. Default: 'Proxy-Connection/ /Lock-Token/ /Content-Range/ /Translate/ /via/ /if/'. +- `secpcrematchlimit`: Sets the number for the match limit in the PCRE library. Default: '1500' +- `secpcrematchlimitrecursion`: Sets the number for the match limit recursion in the PCRE library. Default: '1500' ##### Class: `apache::mod::wsgi` @@ -1644,7 +1692,7 @@ Otherwise, Puppet follows it literally. - `package_name`: Names the package that installs `mod_wsgi`. Default: undef. - `wsgi_python_home`: Defines the [`WSGIPythonHome`][] directive, such as '/path/to/venv'. Valid options: path. Default: undef. - `wsgi_python_path`: Defines the [`WSGIPythonPath`][] directive, such as '/path/to/venv/site-packages'. Valid options: path. Default: undef. -- `wsgi_socket_prefix`: Defines the [`WSGISocketPrefix`][] directive, such as "\${APACHE_RUN_DIR}WSGI". Default: `wsgi_socket_prefix` in [`apache::params`][]. +- `wsgi_socket_prefix`: Defines the [`WSGISocketPrefix`][] directive, such as "\${APACHE\_RUN\_DIR}WSGI". Default: `wsgi_socket_prefix` in [`apache::params`][]. The class's parameters correspond to the module's directives. See the [module's documentation][`mod_wsgi`] for details. @@ -1736,7 +1784,7 @@ Specifies whether the configuration file should be present. Valid options: 'abse ##### `confdir` -Sets the directory in which Puppet places configuration files. Default: '$::apache::confd_dir'. +Sets the directory in which Puppet places configuration files. Default: '$::apache::confd\_dir'. ##### `content` @@ -1798,7 +1846,7 @@ Sets the [MIME `content-type`][] of the file to be processed by the FastCGI serv #### Define: `apache::listen` -Adds [`Listen`][] directives to `ports.conf` in the Apache configuration directory that define the Apache server's or a virtual host's listening address and port. The [`apache::vhost`][] class uses this define, and titles take the form '', ':', or ':'. +Adds [`Listen`][] directives to `ports.conf` in the Apache configuration directory that define the Apache server's or a virtual host's listening address and port. The [`apache::vhost`][] class uses this define, and titles take the form '\', '\:\', or '\:\'. #### Define: `apache::mod` @@ -1838,7 +1886,7 @@ Specifies a path to the module. Default: [`lib_path`][]/[`lib`][]. Don't manuall #### Define: `apache::namevirtualhost` -Enables [name-based virtual hosts][] and adds all related directives to the `ports.conf` file in the Apache HTTPD configuration directory. Titles can take the forms '\*', '*:', '\_default_:, '', or ':'. +Enables [name-based virtual hosts][] and adds all related directives to the `ports.conf` file in the Apache HTTPD configuration directory. Titles can take the forms '\*', '\*:\', '\_default\_:\, '\', or '\:\'. #### Define: `apache::vhost` @@ -1862,7 +1910,7 @@ Specifies that only requests with particular environment variables be logged. De ##### `access_log_file` -Sets the filename of the `*_access.log` placed in [`logroot`][]. Given a virtual host---for instance, example.com---it defaults to 'example.com_ssl.log' for [SSL-encrypted][SSL encryption] virtual hosts and 'example.com_access.log' for unencrypted virtual hosts. +Sets the filename of the `*_access.log` placed in [`logroot`][]. Given a virtual host---for instance, example.com---it defaults to 'example.com\_ssl.log' for [SSL-encrypted][SSL encryption] virtual hosts and 'example.com\_access.log' for unencrypted virtual hosts. ##### `access_log_format` @@ -1900,7 +1948,7 @@ Passes a list of [Hashes][Hash] to the virtual host to create [`Alias`][], [`Ali For example: -~~~ puppet +``` puppet aliases => [ { aliasmatch => '^/image/(.*)\.jpg$', path => '/files/jpg.images/$1.jpg', @@ -1918,7 +1966,7 @@ aliases => [ path => '/usr/share/nagios/html', }, ], -~~~ +``` For the `alias`, `aliasmatch`, `scriptalias` and `scriptaliasmatch` keys to work, each needs a corresponding context, such as `` or ``. Puppet creates the directives in the order specified in the `aliases` parameter. As described in the [`mod_alias`][] documentation, add more specific `alias`, `aliasmatch`, `scriptalias` or `scriptaliasmatch` parameters before the more general ones to avoid shadowing. @@ -1978,7 +2026,7 @@ Specifies whether `*_error.log` directives should be configured. Defaults to 'tr ##### `error_log_file` -Points to the `*_error.log` file. Given a vhost, example.com, it defaults to 'example.com_ssl_error.log' for SSL vhosts and 'example.com_access_error.log' for non-SSL vhosts. +Points to the `*_error.log` file. Given a vhost, example.com, it defaults to 'example.com\_ssl\_error.log' for SSL vhosts and 'example.com_access_error.log' for non-SSL vhosts. ##### `error_log_pipe` @@ -1992,14 +2040,14 @@ Sends all error log messages to syslog. Defaults to 'undef'. A list of hashes which can be used to override the [ErrorDocument](https://httpd.apache.org/docs/current/mod/core.html#errordocument) settings for this vhost. Defaults to '[]'. Example: -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': error_documents => [ { 'error_code' => '503', 'document' => '/service-unavail' }, { 'error_code' => '407', 'document' => 'https://example.com/proxy/login' }, ], } -~~~ +``` ##### `ensure` @@ -2013,7 +2061,7 @@ Sets the [FallbackResource](http://httpd.apache.org/docs/current/mod/mod_dir.htm [Filters](http://httpd.apache.org/docs/2.2/mod/mod_filter.html) enable smart, context-sensitive configuration of output content filters. -~~~ puppet +``` puppet apache::vhost { "$::fqdn": filters => [ 'FilterDeclare COMPRESS', @@ -2022,7 +2070,7 @@ Sets the [FallbackResource](http://httpd.apache.org/docs/current/mod/mod_dir.htm 'FilterProtocol COMPRESS DEFLATE change=yes;byteranges=no', ], } -~~~ +``` ##### `force_type` @@ -2054,7 +2102,7 @@ Configures [ITK](http://mpm-itk.sesse.net/) in a hash. Keys can be: Usage typically looks like: -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', itk => { @@ -2062,7 +2110,7 @@ apache::vhost { 'sample.example.net': group => 'somegroup', }, } -~~~ +``` ##### `auth_kerb` @@ -2070,7 +2118,7 @@ Enable [`mod_auth_kerb`][] parameters for a virtual host. Valid values are 'true Usage typically looks like: -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': auth_kerb => true, krb_method_negotiate => 'on', @@ -2083,7 +2131,7 @@ apache::vhost { 'sample.example.net': auth_require => 'valid-user', }, } -~~~ +``` Related parameters follow the names of `mod_auth_kerb` directives: @@ -2108,7 +2156,7 @@ This option enables credential saving functionality. Default is 'off' ##### `logroot` -Specifies the location of the virtual host's logfiles. Defaults to '/var/log//'. +Specifies the location of the virtual host's logfiles. Defaults to '/var/log/\/'. ##### `$logroot_ensure` @@ -2135,17 +2183,17 @@ Boolean. Only valid if apache::mod::security is included. Used to disable mod_ Array of mod_security IDs to remove from the vhost. Also takes a hash allowing removal of an ID from a specific location. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': modsec_disable_ids => [ 90015, 90016 ], } -~~~ +``` -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': modsec_disable_ids => { '/location1' => [ 90015, 90016 ] }, } -~~~ +``` ###### `modsec_disable_ips` @@ -2173,12 +2221,12 @@ Sets the [ProxyErrorOverride Directive](http://httpd.apache.org/docs/current/mod Sets the [`Options`][] for the specified virtual host. Default: ['Indexes','FollowSymLinks','MultiViews'], as demonstrated below: -~~~ puppet +``` puppet apache::vhost { 'site.name.fdqn': … options => ['Indexes','FollowSymLinks','MultiViews'], } -~~~ +``` **Note**: If you use the [`directories`][] parameter of [`apache::vhost`][], 'Options', 'Override', and 'DirectoryIndex' are ignored because they are parameters within `directories`. @@ -2242,15 +2290,17 @@ Specifies the destination address of a [ProxyPass](http://httpd.apache.org/docs/ ##### `proxy_pass` -Specifies an array of `path => URI` for a [ProxyPass](http://httpd.apache.org/docs/current/mod/mod_proxy.html#proxypass) configuration. Defaults to 'undef'. Optionally parameters can be added as an array. +Specifies an array of `path => URI` for a [ProxyPass](http://httpd.apache.org/docs/current/mod/mod_proxy.html#proxypass) configuration. Defaults to 'undef'. Optionally parameters and location options can be added as an array. -~~~ puppet +``` puppet apache::vhost { 'site.name.fdqn': … proxy_pass => [ { 'path' => '/a', 'url' => 'http://backend-a/' }, { 'path' => '/b', 'url' => 'http://backend-b/' }, { 'path' => '/c', 'url' => 'http://backend-a/c', 'params' => {'max'=>20, 'ttl'=>120, 'retry'=>300}}, + { 'path' => '/c', 'url' => 'http://backend-a/c', + 'options' => {'Require'=>'valid-user', 'AuthType'=>'Kerberos', 'AuthName'=>'Kerberos Login'}}, { 'path' => '/l', 'url' => 'http://backend-xy', 'reverse_urls' => ['http://backend-x', 'http://backend-y'] }, { 'path' => '/d', 'url' => 'http://backend-a/d', @@ -2260,13 +2310,13 @@ apache::vhost { 'site.name.fdqn': { 'path' => '/f', 'url' => 'http://backend-f/', 'setenv' => ['proxy-nokeepalive 1','force-proxy-request-1.0 1']}, { 'path' => '/g', 'url' => 'http://backend-g/', - 'reverse_cookies' => [{'path' => '/g', 'url' => 'http://backend-g/',}], }, + 'reverse_cookies' => [{'path' => '/g', 'url' => 'http://backend-g/',}, {'domain' => 'http://backend-g', 'url' => 'http:://backend-g',},], }, ], } -~~~ +``` `reverse_urls` is optional and can be an array or a string. It is useful when used with `mod_proxy_balancer`. -`reverse_cookies` is optional and is used to set ProxyPassReverseCookiePath. +`reverse_cookies` is optional and is used to set ProxyPassReverseCookiePath and/or ProxyPassReverseCookieDomain. `params` is an optional parameter. It allows to provide the ProxyPass key=value parameters (Connection settings). `setenv` is optional and is an array to set environment variables for the proxy directive, for details see http://httpd.apache.org/docs/current/mod/mod_proxy.html#envsettings @@ -2284,11 +2334,11 @@ This directive is equivalent to proxy_pass, but takes regular expressions, see [ ##### `rack_base_uris` -Specifies the resource identifiers for a rack configuration. The file paths specified are listed as rack application roots for [Phusion Passenger](http://www.modrails.com/documentation/Users%20guide%20Apache.html#_railsbaseuri_and_rackbaseuri) in the _rack.erb template. Defaults to 'undef'. +Specifies the resource identifiers for a rack configuration. The file paths specified are listed as rack application roots for [Phusion Passenger](http://www.modrails.com/documentation/Users%20guide%20Apache.html#_railsbaseuri_and_rackbaseuri) in the \_rack.erb template. Defaults to 'undef'. #####`passenger_base_uris` -Used to specify that the given URI is a Phusion Passenger-served application. The file paths specified are listed as passenger application roots for [Phusion Passenger](https://www.phusionpassenger.com/documentation/Users%20guide%20Apache.html#PassengerBaseURI) in the _passenger_base_uris.erb template. Defaults to 'undef'. +Used to specify that the given URI is a Phusion Passenger-served application. The file paths specified are listed as passenger application roots for [Phusion Passenger](https://www.phusionpassenger.com/documentation/Users%20guide%20Apache.html#PassengerBaseURI) in the \_passenger\_base\_uris.erb template. Defaults to 'undef'. ##### `redirect_dest` @@ -2298,43 +2348,43 @@ Specifies the address to redirect to. Defaults to 'undef'. Specifies the source URIs that redirect to the destination specified in `redirect_dest`. If more than one item for redirect is supplied, the source and destination must be the same length, and the items are order-dependent. -~~~ puppet +``` puppet apache::vhost { 'site.name.fdqn': … redirect_source => ['/images','/downloads'], redirect_dest => ['http://img.example.com/','http://downloads.example.com/'], } -~~~ +``` ##### `redirect_status` Specifies the status to append to the redirect. Defaults to 'undef'. -~~~ puppet +``` puppet apache::vhost { 'site.name.fdqn': … redirect_status => ['temp','permanent'], } -~~~ +``` ##### `redirectmatch_regexp` & `redirectmatch_status` & `redirectmatch_dest` Determines which server status should be raised for a given regular expression and where to forward the user to. Entered as arrays. Defaults to 'undef'. -~~~ puppet +``` puppet apache::vhost { 'site.name.fdqn': … redirectmatch_status => ['404','404'], redirectmatch_regexp => ['\.git(/.*|$)/','\.svn(/.*|$)'], redirectmatch_dest => ['http://www.example.com/1','http://www.example.com/2'], } -~~~ +``` ##### `request_headers` Modifies collected [request headers](http://httpd.apache.org/docs/current/mod/mod_headers.html#requestheader) in various ways, including adding additional request headers, removing request headers, etc. Defaults to 'undef'. -~~~ puppet +``` puppet apache::vhost { 'site.name.fdqn': … request_headers => [ @@ -2342,23 +2392,23 @@ apache::vhost { 'site.name.fdqn': 'unset MirrorID', ], } -~~~ +``` ##### `rewrites` Creates URL rewrite rules. Expects an array of hashes, and the hash keys can be any of 'comment', 'rewrite_base', 'rewrite_cond', 'rewrite_rule' or 'rewrite_map'. Defaults to 'undef'. For example, you can specify that anyone trying to access index.html is served welcome.html -~~~ puppet +``` puppet apache::vhost { 'site.name.fdqn': … rewrites => [ { rewrite_rule => ['^index\.html$ welcome.html'] } ] } -~~~ +``` The parameter allows rewrite conditions that, when true, execute the associated rule. For instance, if you wanted to rewrite URLs only if the visitor is using IE -~~~ puppet +``` puppet apache::vhost { 'site.name.fdqn': … rewrites => [ @@ -2369,11 +2419,11 @@ apache::vhost { 'site.name.fdqn': }, ], } -~~~ +``` You can also apply multiple conditions. For instance, rewrite index.html to welcome.html only when the browser is Lynx or Mozilla (version 1 or 2) -~~~ puppet +``` puppet apache::vhost { 'site.name.fdqn': … rewrites => [ @@ -2384,11 +2434,11 @@ apache::vhost { 'site.name.fdqn': }, ], } -~~~ +``` Multiple rewrites and conditions are also possible -~~~ puppet +``` puppet apache::vhost { 'site.name.fdqn': … rewrites => [ @@ -2413,7 +2463,7 @@ apache::vhost { 'site.name.fdqn': }, ], } -~~~ +``` Refer to the [`mod_rewrite` documentation](http://httpd.apache.org/docs/current/mod/mod_rewrite.html) for more details on what is possible with rewrite rules and conditions. @@ -2427,7 +2477,7 @@ Defines a directory of CGI scripts to be aliased to the path '/cgi-bin', for exa Passes an array of hashes to the vhost to create either ScriptAlias or ScriptAliasMatch statements per the [`mod_alias` documentation](http://httpd.apache.org/docs/current/mod/mod_alias.html). -~~~ puppet +``` puppet scriptaliases => [ { alias => '/myscript', @@ -2446,7 +2496,7 @@ scriptaliases => [ path => '/usr/share/neatscript', }, ] -~~~ +``` The ScriptAlias and ScriptAliasMatch directives are created in the order specified. As with [Alias and AliasMatch](#aliases) directives, specify more specific aliases before more general ones to avoid shadowing. @@ -2468,11 +2518,11 @@ Used by HTTPD to set environment variables for vhosts. Defaults to '[]'. Example: -~~~ puppet +``` puppet apache::vhost { 'setenv.example.com': setenv => ['SPECIAL_PATH /foo/bin'], } -~~~ +``` ##### `setenvif` @@ -2490,7 +2540,7 @@ Set up a virtual host with [suPHP](http://suphp.org/DocumentationView.html?file= To set up a virtual host with suPHP -~~~ puppet +``` puppet apache::vhost { 'suphp.example.com': port => '80', docroot => '/home/appuser/myphpapp', @@ -2501,17 +2551,17 @@ apache::vhost { 'suphp.example.com': 'suphp' => { user => 'myappuser', group => 'myappgroup' }, } } -~~~ +``` ##### `vhost_name` -Enables name-based virtual hosting. If no IP is passed to the virtual host, but the vhost is assigned a port, then the vhost name is 'vhost_name:port'. If the virtual host has no assigned IP or port, the vhost name is set to the title of the resource. Defaults to '*'. +Enables name-based virtual hosting. If no IP is passed to the virtual host, but the vhost is assigned a port, then the vhost name is 'vhost_name:port'. If the virtual host has no assigned IP or port, the vhost name is set to the title of the resource. Defaults to '\*'. ##### `virtual_docroot` Sets up a virtual host with a wildcard alias subdomain mapped to a directory with the same name. For example, 'http://example.com' would map to '/var/www/example.com'. Defaults to 'false'. -~~~ puppet +``` puppet apache::vhost { 'subdomain.loc': vhost_name => '*', port => '80', @@ -2519,7 +2569,7 @@ apache::vhost { 'subdomain.loc': docroot => '/var/www', serveraliases => ['*.loc',], } -~~~ +``` ##### `wsgi_daemon_process`, `wsgi_daemon_process_options`, `wsgi_process_group`, `wsgi_script_aliases`, & `wsgi_pass_authorization` @@ -2539,7 +2589,7 @@ Set up a virtual host with [WSGI](https://code.google.com/p/modwsgi/). To set up a virtual host with WSGI -~~~ puppet +``` puppet apache::vhost { 'wsgi.example.com': port => '80', docroot => '/var/www/pythonapp', @@ -2553,19 +2603,19 @@ apache::vhost { 'wsgi.example.com': wsgi_script_aliases => { '/' => '/var/www/demo.wsgi' }, wsgi_chunked_request => 'On', } -~~~ +``` #### Parameter `directories` for `apache::vhost` -The `directories` parameter within the `apache::vhost` class passes an array of hashes to the vhost to create [Directory](http://httpd.apache.org/docs/current/mod/core.html#directory), [File](http://httpd.apache.org/docs/current/mod/core.html#files), and [Location](http://httpd.apache.org/docs/current/mod/core.html#location) directive blocks. These blocks take the form, '< Directory /path/to/directory>...< /Directory>'. +The `directories` parameter within the `apache::vhost` class passes an array of hashes to the vhost to create [Directory](http://httpd.apache.org/docs/current/mod/core.html#directory), [File](http://httpd.apache.org/docs/current/mod/core.html#files), and [Location](http://httpd.apache.org/docs/current/mod/core.html#location) directive blocks. These blocks take the form, '\< Directory /path/to/directory\>...\'. The `path` key sets the path for the directory, files, and location blocks. Its value must be a path for the 'directory', 'files', and 'location' providers, or a regex for the 'directorymatch', 'filesmatch', or 'locationmatch' providers. Each hash passed to `directories` **must** contain `path` as one of the keys. -The `provider` key is optional. If missing, this key defaults to 'directory'. Valid values for `provider` are 'directory', 'files', 'location', 'directorymatch', 'filesmatch', or 'locationmatch'. If you set `provider` to 'directorymatch', it uses the keyword 'DirectoryMatch' in the Apache config file. +The `provider` key is optional. If missing, this key defaults to 'directory'. Valid values for `provider` are 'directory', 'files', 'proxy', 'location', 'directorymatch', 'filesmatch', 'proxymatch' or 'locationmatch'. If you set `provider` to 'directorymatch', it uses the keyword 'DirectoryMatch' in the Apache config file. General `directories` usage looks something like -~~~ puppet +``` puppet apache::vhost { 'files.example.net': docroot => '/var/www/files', directories => [ @@ -2575,18 +2625,18 @@ apache::vhost { 'files.example.net': }, ], } -~~~ +``` *Note:* At least one directory should match the `docroot` parameter. After you start declaring directories, `apache::vhost` assumes that all required Directory blocks will be declared. If not defined, a single default Directory block is created that matches the `docroot` parameter. Available handlers, represented as keys, should be placed within the `directory`, `files`, or `location` hashes. This looks like -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ { path => '/path/to/directory', handler => value } ], } -~~~ +``` Any handlers you do not set in these hashes are considered 'undefined' within Puppet and are not added to the virtual host, resulting in the module using their default values. Supported handlers are: @@ -2594,7 +2644,7 @@ Any handlers you do not set in these hashes are considered 'undefined' within Pu Sets [AddHandler](http://httpd.apache.org/docs/current/mod/mod_mime.html#addhandler) directives, which map filename extensions to the specified handler. Accepts a list of hashes, with `extensions` serving to list the extensions being managed by the handler, and takes the form: `{ handler => 'handler-name', extensions => ['extension']}`. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2603,13 +2653,13 @@ apache::vhost { 'sample.example.net': }, ], } -~~~ +``` ###### `allow` Sets an [Allow](http://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#allow) directive, which groups authorizations based on hostnames or IPs. **Deprecated:** This parameter is being deprecated due to a change in Apache. It only works with Apache 2.2 and lower. You can use it as a single string for one rule or as an array for more than one. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2618,13 +2668,13 @@ apache::vhost { 'sample.example.net': }, ], } -~~~ +``` ###### `allow_override` Sets the types of directives allowed in [.htaccess](http://httpd.apache.org/docs/current/mod/core.html#allowoverride) files. Accepts an array. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2633,7 +2683,7 @@ apache::vhost { 'sample.example.net': }, ], } -~~~ +``` ###### `auth_basic_authoritative` @@ -2695,7 +2745,7 @@ Sets the value for [AuthUserFile](http://httpd.apache.org/docs/current/mod/mod_a Pass a string of custom configuration directives to be placed at the end of the directory configuration. -~~~ puppet +``` puppet apache::vhost { 'monitor': … directories => [ @@ -2716,13 +2766,13 @@ Pass a string of custom configuration directives to be placed at the end of the }, ] } -~~~ +``` ###### `deny` Sets a [Deny](http://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#deny) directive, specifying which hosts are denied access to the server. **Deprecated:** This parameter is being deprecated due to a change in Apache. It only works with Apache 2.2 and lower. You can use it as a single string for one rule or as an array for more than one. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2731,13 +2781,13 @@ Sets a [Deny](http://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#deny) dir }, ], } -~~~ +``` ###### `error_documents` An array of hashes used to override the [ErrorDocument](https://httpd.apache.org/docs/current/mod/core.html#errordocument) settings for the directory. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': directories => [ { path => '/srv/www', @@ -2749,14 +2799,14 @@ apache::vhost { 'sample.example.net': }, ], } -~~~ +``` ###### `ext_filter_options` Sets the [ExtFilterOptions](https://httpd.apache.org/docs/current/mod/mod_ext_filter.html) directive. Note that you must declare `class { 'apache::mod::ext_filter': }` before using this directive. -~~~ puppet +``` puppet apache::vhost { 'filter.example.org': docroot => '/var/www/filter', directories => [ @@ -2765,14 +2815,14 @@ apache::vhost { 'filter.example.org': }, ], } -~~~ +``` ###### `geoip_enable` Sets the [GeoIPEnable](http://dev.maxmind.com/geoip/legacy/mod_geoip2/#Configuration) directive. Note that you must declare `class {'apache::mod::geoip': }` before using this directive. -~~~ puppet +``` puppet apache::vhost { 'first.example.com': docroot => '/var/www/first', directories => [ @@ -2781,13 +2831,13 @@ apache::vhost { 'first.example.com': }, ], } -~~~ +``` ###### `headers` Adds lines for [Header](http://httpd.apache.org/docs/current/mod/mod_headers.html#header) directives. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => { @@ -2795,13 +2845,13 @@ apache::vhost { 'sample.example.net': headers => 'Set X-Robots-Tag "noindex, noarchive, nosnippet"', }, } -~~~ +``` ###### `index_options` Allows configuration settings for [directory indexing](http://httpd.apache.org/docs/current/mod/mod_autoindex.html#indexoptions). -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2812,13 +2862,13 @@ apache::vhost { 'sample.example.net': }, ], } -~~~ +``` ###### `index_order_default` Sets the [default ordering](https://httpd.apache.org/docs/current/mod/mod_autoindex.html#indexorderdefault) of the directory index. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2828,13 +2878,13 @@ apache::vhost { 'sample.example.net': }, ], } -~~~ +``` ###### `index_style_sheet` Sets the [IndexStyleSheet](https://httpd.apache.org/docs/current/mod/mod_autoindex.html#indexstylesheet), which adds a CSS stylesheet to the directory index. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2845,13 +2895,13 @@ apache::vhost { 'sample.example.net': }, ], } -~~~ +``` ###### `mellon_enable` Sets the [MellonEnable][`mod_auth_mellon`] directory to enable [`mod_auth_melon`][]. You can use [`apache::mod::auth_mellon`][] to install `mod_auth_mellon`. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2873,7 +2923,7 @@ apache::vhost { 'sample.example.net': }, ] } -~~~ +``` Related parameters follow the names of `mod_auth_melon` directives: @@ -2891,7 +2941,7 @@ to environment variables. Lists the [Options](https://httpd.apache.org/docs/current/mod/core.html#options) for the given Directory block. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2900,13 +2950,13 @@ apache::vhost { 'sample.example.net': }, ], } -~~~ +``` ###### `order` Sets the order of processing Allow and Deny statements as per [Apache core documentation](http://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#order). **Deprecated:** This parameter is being deprecated due to a change in Apache. It only works with Apache 2.2 and lower. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2915,13 +2965,13 @@ apache::vhost { 'sample.example.net': }, ], } -~~~ +``` ###### `passenger_enabled` Sets the value for the [PassengerEnabled](http://www.modrails.com/documentation/Users%20guide%20Apache.html#PassengerEnabled) directive to 'on' or 'off'. Requires `apache::mod::passenger` to be included. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2930,7 +2980,7 @@ apache::vhost { 'sample.example.net': }, ], } -~~~ +``` **Note:** There is an [issue](http://www.conandalton.net/2010/06/passengerenabled-off-not-working.html) using the PassengerEnabled directive with the PassengerHighPerformance directive. @@ -2948,7 +2998,7 @@ apache::vhost { 'sample.example.net': Sets a `Require` directive as per the [Apache Authz documentation](http://httpd.apache.org/docs/current/mod/mod_authz_core.html#require). If no `require` is set, it will default to `Require all granted`. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2957,11 +3007,11 @@ Sets a `Require` directive as per the [Apache Authz documentation](http://httpd. } ], } -~~~ +``` If `require` is set to `unmanaged` it will not be set at all. This is useful for complex authentication/authorization requirements which are handled in a custom fragment. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2970,13 +3020,13 @@ If `require` is set to `unmanaged` it will not be set at all. This is useful for } ], } -~~~ +``` ###### `satisfy` Sets a `Satisfy` directive per the [Apache Core documentation](http://httpd.apache.org/docs/2.2/mod/core.html#satisfy). **Deprecated:** This parameter is deprecated due to a change in Apache and only works with Apache 2.2 and lower. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -2985,13 +3035,13 @@ apache::vhost { 'sample.example.net': } ], } -~~~ +``` ###### `sethandler` Sets a `SetHandler` directive per the [Apache Core documentation](http://httpd.apache.org/docs/2.2/mod/core.html#sethandler). -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': docroot => '/path/to/directory', directories => [ @@ -3000,13 +3050,13 @@ apache::vhost { 'sample.example.net': } ], } -~~~ +``` ###### `set_output_filter` Sets a `SetOutputFilter` directive per the [Apache Core documentation](http://httpd.apache.org/docs/current/mod/core.html#setoutputfilter). -~~~ puppet +``` puppet apache::vhost{ 'filter.example.net': docroot => '/path/to/directory', directories => [ @@ -3015,13 +3065,13 @@ apache::vhost{ 'filter.example.net': }, ], } -~~~ +``` ###### `rewrites` Creates URL [`rewrites`](#rewrites) rules in vhost directories. Expects an array of hashes, and the hash keys can be any of 'comment', 'rewrite_base', 'rewrite_cond', or 'rewrite_rule'. -~~~ puppet +``` puppet apache::vhost { 'secure.example.net': docroot => '/path/to/directory', directories => [ @@ -3040,15 +3090,15 @@ apache::vhost { 'secure.example.net': }, ], } -~~~ +``` -***Note**: If you include rewrites in your directories, also include `apache::mod::rewrite` and consider setting the rewrites using the `rewrites` parameter in `apache::vhost` rather than setting the rewrites in the vhost directories. +**Note**: If you include rewrites in your directories, also include `apache::mod::rewrite` and consider setting the rewrites using the `rewrites` parameter in `apache::vhost` rather than setting the rewrites in the vhost directories. ###### `shib_request_setting` Allows a valid content setting to be set or altered for the application request. This command takes two parameters: the name of the content setting, and the value to set it to. Check the Shibboleth [content setting documentation](https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPContentSettings) for valid settings. This key is disabled if `apache::mod::shib` is not defined. Check the [`mod_shib` documentation](https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPApacheConfig#NativeSPApacheConfig-Server/VirtualHostOptions) for more details. -~~~ puppet +``` puppet apache::vhost { 'secure.example.net': docroot => '/path/to/directory', directories => [ @@ -3058,7 +3108,7 @@ apache::vhost { 'secure.example.net': }, ], } -~~~ +``` ###### `shib_use_headers` @@ -3068,7 +3118,7 @@ When set to 'On', this turns on the use of request headers to publish attributes String or list of [SSLOptions](https://httpd.apache.org/docs/current/mod/mod_ssl.html#ssloptions), which configure SSL engine run-time options. This handler takes precedence over SSLOptions set in the parent block of the vhost. -~~~ puppet +``` puppet apache::vhost { 'secure.example.net': docroot => '/path/to/directory', directories => [ @@ -3080,13 +3130,13 @@ apache::vhost { 'secure.example.net': }, ], } -~~~ +``` ###### `suphp` A hash containing the 'user' and 'group' keys for the [suPHP_UserGroup](http://www.suphp.org/DocumentationView.html?file=apache/CONFIG) setting. It must be used with `suphp_engine => on` in the vhost declaration, and can only be passed within `directories`. -~~~ puppet +``` puppet apache::vhost { 'secure.example.net': docroot => '/path/to/directory', directories => [ @@ -3098,7 +3148,7 @@ apache::vhost { 'secure.example.net': }, ], } -~~~ +``` #### SSL parameters for `apache::vhost` @@ -3154,44 +3204,48 @@ Specifies the SSL key. Defaults are based on your operating system: '/etc/pki/tl ##### `ssl_verify_client` -Sets the [SSLVerifyClient](http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslverifyclient) directive, which sets the certificate verification level for client authentication. Valid values are: 'none', 'optional', 'require', and 'optional_no_ca'. Defaults to 'undef'. +Sets the [SSLVerifyClient](http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslverifyclient) directive, which sets the certificate verification level for client authentication. Valid values are: 'none', 'optional', 'require', and 'optional\_no\_ca'. Defaults to 'undef'. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': … ssl_verify_client => 'optional', } -~~~ +``` ##### `ssl_verify_depth` Sets the [SSLVerifyDepth](http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslverifydepth) directive, which specifies the maximum depth of CA certificates in client certificate verification. Defaults to 'undef'. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': … ssl_verify_depth => 1, } -~~~ +``` + +##### `ssl_proxy_verify` + +Sets the [SSLProxyVerify](http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyverify) directive, which configures certificate verification of the remote server when a proxy is configured to forward requests to a remote SSL server. Defaults to 'undef'. ##### `ssl_proxy_machine_cert` Sets the [SSLProxyMachineCertificateFile](http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxymachinecertificatefile) directive, which specifies an all-in-one file where you keep the certs and keys used for this server to authenticate itself to remote servers. This file should be a concatenation of the PEM-encoded certificate files in order of preference. Defaults to 'undef'. -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': … ssl_proxy_machine_cert => '/etc/httpd/ssl/client_certificate.pem', } -~~~ +``` ##### `ssl_proxy_check_peer_cn` - + Sets the [SSLProxyMachinePeerCN](http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxycheckpeercn) directive, which specified whether the remote server certificate's CN field is compared against the hostname of the request URL . Defaults to 'undef'. ##### `ssl_proxy_check_peer_name` - + Sets the [SSLProxyMachinePeerName](http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxycheckpeername) directive, which specified whether the remote server certificate's CN field is compared against the hostname of the request URL . Defaults to 'undef'. ##### `ssl_options` @@ -3200,21 +3254,21 @@ Sets the [SSLOptions](http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslo A string: -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': … ssl_options => '+ExportCertData', } -~~~ +``` An array: -~~~ puppet +``` puppet apache::vhost { 'sample.example.net': … ssl_options => [ '+StrictRequire', '+ExportCertData' ], } -~~~ +``` ##### `ssl_openssl_conf_cmd` @@ -3226,13 +3280,13 @@ Specifies whether or not to use [SSLProxyEngine](http://httpd.apache.org/docs/cu ####Define: FastCGI Server -This type is intended for use with mod_fastcgi. It allows you to define one or more external FastCGI servers to handle specific file types. +This type is intended for use with mod\_fastcgi. It allows you to define one or more external FastCGI servers to handle specific file types. ** Note ** If using Ubuntu 10.04+, you'll need to manually enable the multiverse repository. Ex: -~~~ puppet +``` puppet apache::fastcgi::server { 'php': host => '127.0.0.1:9000', timeout => 15, @@ -3241,17 +3295,17 @@ apache::fastcgi::server { 'php': fcgi_alias => '/php.fcgi', file_type => 'application/x-httpd-php' } -~~~ +``` Within your virtual host, you can then configure the specified file type to be handled by the fastcgi server specified above. -~~~ puppet +``` puppet apache::vhost { 'www': ... custom_fragment => 'AddType application/x-httpd-php .php' ... } -~~~ +``` ##### `host` @@ -3263,7 +3317,7 @@ The number of seconds of FastCGI application inactivity allowed before the reque ##### `flush` -Force a write to the client as data is received from the application. By default, mod_fastcgi buffers data in order to free the application as quickly as possible. +Force a write to the client as data is received from the application. By default, mod\_fastcgi buffers data in order to free the application as quickly as possible. ##### `faux_path` @@ -3321,22 +3375,6 @@ The Apache module relies heavily on templates to enable the [`apache::vhost`][] ## Limitations -### Ubuntu 10.04 - -The [`apache::vhost::WSGIImportScript`][] parameter creates a statement inside the virtual host that is unsupported on older versions of Apache, causing it to fail. This will be remedied in a future refactoring. - -### RHEL/CentOS 5 - -The [`apache::mod::passenger`][] and [`apache::mod::proxy_html`][] classes are untested since repositories are missing compatible packages. - -### RHEL/CentOS 6 - -The [`apache::mod::passenger`][] class is not installing as the the EL6 repository is missing compatible packages. - -### RHEL/CentOS 7 - -The [`apache::mod::passenger`][] class is untested as the EL7 repository is missing compatible packages, which also blocks us from testing the [`apache::vhost`][] define's [`rack_base_uris`][] parameter. - ### General This module is CI tested against both [open source Puppet][] and [Puppet Enterprise][] on: @@ -3348,13 +3386,22 @@ This module is CI tested against both [open source Puppet][] and [Puppet Enterpr This module also provides functions for other distributions and operating systems, such as FreeBSD, Gentoo, and Amazon Linux, but is not formally tested on them and are subject to regressions. +### Ubuntu 10.04 + +The [`apache::vhost::wsgi_import_script`][] parameter creates a statement inside the virtual host that is unsupported on older versions of Apache, causing it to fail. This will be remedied in a future refactoring. + +### RHEL/CentOS +The [`apache::mod::auth_cas`][], [`apache::mod::passenger`][], [`apache::mod::proxy_html`][] and [`apache::mod::shib`][] classes are not functional on RH/CentOS without providing dependency packages from extra repositories. + +See their respective documentation above for related repositories and packages. + ### SELinux and custom paths If [SELinux][] is in [enforcing mode][] and you want to use custom paths for `logroot`, `mod_dir`, `vhost_dir`, and `docroot`, you need to manage the files' context yourself. You can do this with Puppet: -~~~ puppet +``` puppet exec { 'set_apache_defaults': command => 'semanage fcontext -a -t httpd_sys_content_t "/custom/path(/.*)?"', path => '/bin:/usr/bin/:/sbin:/usr/sbin', @@ -3391,7 +3438,7 @@ apache::vhost { 'test.server': docroot => '/custom/path', additional_includes => '/custom/path/include', } -~~~ +``` You need to set the contexts using `semanage fcontext` instead of `chcon` because Puppet's `file` resources reset the values' context in the database if the resource doesn't specify it. @@ -3415,18 +3462,18 @@ This project contains tests for both [rspec-puppet][] and [beaker-rspec][] to ve #### Testing quickstart: Ruby > 1.8.7 -~~~ +``` gem install bundler bundle install bundle exec rake spec bundle exec rspec spec/acceptance RS_DEBUG=yes bundle exec rspec spec/acceptance -~~~ +``` #### Testing quickstart: Ruby = 1.8.7 -~~~ +``` gem install bundler bundle install --without system_tests bundle exec rake spec -~~~ +``` diff --git a/apache/Rakefile b/apache/Rakefile index 416807dad..636508b00 100644 --- a/apache/Rakefile +++ b/apache/Rakefile @@ -1,5 +1,6 @@ -require 'puppetlabs_spec_helper/rake_tasks' +require 'puppet_blacksmith/rake_tasks' require 'puppet-lint/tasks/puppet-lint' +require 'puppetlabs_spec_helper/rake_tasks' PuppetLint.configuration.fail_on_warnings = true PuppetLint.configuration.send('relative') diff --git a/apache/lib/facter/apache_version.rb b/apache/lib/facter/apache_version.rb new file mode 100644 index 000000000..b45c88834 --- /dev/null +++ b/apache/lib/facter/apache_version.rb @@ -0,0 +1,8 @@ +Facter.add(:apache_version) do + setcode do + if Facter::Util::Resolution.which('apachectl') + apache_version = Facter::Util::Resolution.exec('apachectl -v 2>&1') + %r{^Server version: Apache\/([\w\.]+) \(([\w]+)\)}.match(apache_version)[1] + end + end +end diff --git a/apache/lib/puppet/parser/functions/enclose_ipv6.rb b/apache/lib/puppet/parser/functions/enclose_ipv6.rb index 968bd723e..80ffc3aca 100644 --- a/apache/lib/puppet/parser/functions/enclose_ipv6.rb +++ b/apache/lib/puppet/parser/functions/enclose_ipv6.rb @@ -28,16 +28,18 @@ module Puppet::Parser::Functions result = [] input.each do |val| - begin - ip = IPAddr.new(val) - rescue *rescuable_exceptions - raise(Puppet::ParseError, "enclose_ipv6(): Wrong argument "+ - "given #{val} is not an ip address.") + unless val == '*' + begin + ip = IPAddr.new(val) + rescue *rescuable_exceptions + raise(Puppet::ParseError, "enclose_ipv6(): Wrong argument "+ + "given #{val} is not an ip address.") + end + val = "[#{ip.to_s}]" if ip.ipv6? end - val = "[#{ip.to_s}]" if ip.ipv6? - result = [result,val] + result << val end - return result.flatten.compact + return result.uniq end end diff --git a/apache/manifests/balancer.pp b/apache/manifests/balancer.pp index 765dae629..9b7511a03 100644 --- a/apache/manifests/balancer.pp +++ b/apache/manifests/balancer.pp @@ -49,7 +49,7 @@ concat { $target: owner => '0', group => '0', - mode => '0644', + mode => $::apache::file_mode, notify => Class['Apache::Service'], } diff --git a/apache/manifests/fastcgi/server.pp b/apache/manifests/fastcgi/server.pp index afc7c8860..ec89bf778 100644 --- a/apache/manifests/fastcgi/server.pp +++ b/apache/manifests/fastcgi/server.pp @@ -15,7 +15,7 @@ path => "${::apache::confd_dir}/fastcgi-pool-${name}.conf", owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, content => template('apache/fastcgi/server.erb'), require => Exec["mkdir ${::apache::confd_dir}"], before => File[$::apache::confd_dir], diff --git a/apache/manifests/init.pp b/apache/manifests/init.pp index bb50d0b2e..13eb5f717 100644 --- a/apache/manifests/init.pp +++ b/apache/manifests/init.pp @@ -79,6 +79,7 @@ $use_optional_includes = $::apache::params::use_optional_includes, $use_systemd = $::apache::params::use_systemd, $mime_types_additional = $::apache::params::mime_types_additional, + $file_mode = $::apache::params::file_mode, ) inherits ::apache::params { validate_bool($default_vhost) validate_bool($default_ssl_vhost) @@ -241,7 +242,7 @@ concat { $ports_file: owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, notify => Class['Apache::Service'], require => Package['httpd'], } @@ -273,9 +274,24 @@ $scriptalias = '/var/www/localhost/cgi-bin' $access_log_file = 'access.log' - ::portage::makeconf { 'apache2_modules': - content => $default_mods, + if is_array($default_mods) { + if versioncmp($apache_version, '2.4') >= 0 { + if defined('apache::mod::ssl') { + ::portage::makeconf { 'apache2_modules': + content => concat($default_mods, [ 'authz_core', 'socache_shmcb' ]), + } + } else { + ::portage::makeconf { 'apache2_modules': + content => concat($default_mods, 'authz_core'), + } + } + } else { + ::portage::makeconf { 'apache2_modules': + content => $default_mods, + } + } } + file { [ '/etc/apache2/modules.d/.keep_www-servers_apache-2', '/etc/apache2/vhosts.d/.keep_www-servers_apache-2' @@ -329,7 +345,7 @@ ensure => file, content => template($conf_template), notify => Class['Apache::Service'], - require => [Package['httpd'], File[$ports_file]], + require => [Package['httpd'], Concat[$ports_file]], } # preserve back-wards compatibility to the times when default_mods was diff --git a/apache/manifests/mod.pp b/apache/manifests/mod.pp index abdbfcbed..33b4de1ab 100644 --- a/apache/manifests/mod.pp +++ b/apache/manifests/mod.pp @@ -81,7 +81,7 @@ path => "${mod_dir}/${_loadfile_name}", owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, content => template('apache/mod/load.erb'), require => [ Package['httpd'], @@ -99,7 +99,7 @@ target => "${mod_dir}/${_loadfile_name}", owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, require => [ File[$_loadfile_name], Exec["mkdir ${enable_dir}"], @@ -117,7 +117,7 @@ target => "${mod_dir}/${mod}.conf", owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, require => [ File["${mod}.conf"], Exec["mkdir ${enable_dir}"], @@ -134,7 +134,7 @@ target => "${mod_dir}/${_loadfile_name}", owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, require => [ File[$_loadfile_name], Exec["mkdir ${enable_dir}"], @@ -152,7 +152,7 @@ target => "${mod_dir}/${mod}.conf", owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, require => [ File["${mod}.conf"], Exec["mkdir ${enable_dir}"], diff --git a/apache/manifests/mod/alias.pp b/apache/manifests/mod/alias.pp index 5b59baa01..eac21ba66 100644 --- a/apache/manifests/mod/alias.pp +++ b/apache/manifests/mod/alias.pp @@ -11,6 +11,7 @@ file { 'alias.conf': ensure => file, path => "${::apache::mod_dir}/alias.conf", + mode => $::apache::file_mode, content => template('apache/mod/alias.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/auth_cas.pp b/apache/manifests/mod/auth_cas.pp index 5b13af66a..0d1b9111a 100644 --- a/apache/manifests/mod/auth_cas.pp +++ b/apache/manifests/mod/auth_cas.pp @@ -39,6 +39,7 @@ file { 'auth_cas.conf': ensure => file, path => "${::apache::mod_dir}/auth_cas.conf", + mode => $::apache::file_mode, content => template('apache/mod/auth_cas.conf.erb'), require => [ Exec["mkdir ${::apache::mod_dir}"], ], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/auth_mellon.pp b/apache/manifests/mod/auth_mellon.pp index 79f6ffebb..129441bf4 100644 --- a/apache/manifests/mod/auth_mellon.pp +++ b/apache/manifests/mod/auth_mellon.pp @@ -15,6 +15,7 @@ file { 'auth_mellon.conf': ensure => file, path => "${::apache::mod_dir}/auth_mellon.conf", + mode => $::apache::file_mode, content => template('apache/mod/auth_mellon.conf.erb'), require => [ Exec["mkdir ${::apache::mod_dir}"], ], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/authnz_ldap.pp b/apache/manifests/mod/authnz_ldap.pp index b75369ffc..70d0a6363 100644 --- a/apache/manifests/mod/authnz_ldap.pp +++ b/apache/manifests/mod/authnz_ldap.pp @@ -11,6 +11,7 @@ file { 'authnz_ldap.conf': ensure => file, path => "${::apache::mod_dir}/authnz_ldap.conf", + mode => $::apache::file_mode, content => template('apache/mod/authnz_ldap.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/autoindex.pp b/apache/manifests/mod/autoindex.pp index c0969a814..c8cd0658d 100644 --- a/apache/manifests/mod/autoindex.pp +++ b/apache/manifests/mod/autoindex.pp @@ -4,6 +4,7 @@ file { 'autoindex.conf': ensure => file, path => "${::apache::mod_dir}/autoindex.conf", + mode => $::apache::file_mode, content => template('apache/mod/autoindex.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/cgid.pp b/apache/manifests/mod/cgid.pp index 4094c3281..891cdd75b 100644 --- a/apache/manifests/mod/cgid.pp +++ b/apache/manifests/mod/cgid.pp @@ -23,6 +23,7 @@ file { 'cgid.conf': ensure => file, path => "${::apache::mod_dir}/cgid.conf", + mode => $::apache::file_mode, content => template('apache/mod/cgid.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/dav_fs.pp b/apache/manifests/mod/dav_fs.pp index af037e32d..f652d4c21 100644 --- a/apache/manifests/mod/dav_fs.pp +++ b/apache/manifests/mod/dav_fs.pp @@ -12,6 +12,7 @@ file { 'dav_fs.conf': ensure => file, path => "${::apache::mod_dir}/dav_fs.conf", + mode => $::apache::file_mode, content => template('apache/mod/dav_fs.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/deflate.pp b/apache/manifests/mod/deflate.pp index 0748a54e5..e63eeca47 100644 --- a/apache/manifests/mod/deflate.pp +++ b/apache/manifests/mod/deflate.pp @@ -17,6 +17,7 @@ file { 'deflate.conf': ensure => file, path => "${::apache::mod_dir}/deflate.conf", + mode => $::apache::file_mode, content => template('apache/mod/deflate.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/dir.pp b/apache/manifests/mod/dir.pp index 6243a1bb7..bce05e0a4 100644 --- a/apache/manifests/mod/dir.pp +++ b/apache/manifests/mod/dir.pp @@ -13,6 +13,7 @@ file { 'dir.conf': ensure => file, path => "${::apache::mod_dir}/dir.conf", + mode => $::apache::file_mode, content => template('apache/mod/dir.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/disk_cache.pp b/apache/manifests/mod/disk_cache.pp index 2f0a476fa..051d69894 100644 --- a/apache/manifests/mod/disk_cache.pp +++ b/apache/manifests/mod/disk_cache.pp @@ -32,6 +32,7 @@ file { 'disk_cache.conf': ensure => file, path => "${::apache::mod_dir}/disk_cache.conf", + mode => $::apache::file_mode, content => template('apache/mod/disk_cache.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/event.pp b/apache/manifests/mod/event.pp index 389120cb7..6c70589a3 100644 --- a/apache/manifests/mod/event.pp +++ b/apache/manifests/mod/event.pp @@ -27,7 +27,7 @@ File { owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, } # Template uses: @@ -40,6 +40,7 @@ # - $serverlimit file { "${::apache::mod_dir}/event.conf": ensure => file, + mode => $::apache::file_mode, content => template('apache/mod/event.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/expires.pp b/apache/manifests/mod/expires.pp index 10542916a..1531fc54d 100644 --- a/apache/manifests/mod/expires.pp +++ b/apache/manifests/mod/expires.pp @@ -12,6 +12,7 @@ file { 'expires.conf': ensure => file, path => "${::apache::mod_dir}/expires.conf", + mode => $::apache::file_mode, content => template('apache/mod/expires.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/ext_filter.pp b/apache/manifests/mod/ext_filter.pp index b78abb607..244c2b1da 100644 --- a/apache/manifests/mod/ext_filter.pp +++ b/apache/manifests/mod/ext_filter.pp @@ -15,6 +15,7 @@ file { 'ext_filter.conf': ensure => file, path => "${::apache::mod_dir}/ext_filter.conf", + mode => $::apache::file_mode, content => template('apache/mod/ext_filter.conf.erb'), require => [ Exec["mkdir ${::apache::mod_dir}"], ], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/fastcgi.pp b/apache/manifests/mod/fastcgi.pp index 1f7e5df4f..c4da5b1e6 100644 --- a/apache/manifests/mod/fastcgi.pp +++ b/apache/manifests/mod/fastcgi.pp @@ -14,6 +14,7 @@ file { 'fastcgi.conf': ensure => file, path => "${::apache::mod_dir}/fastcgi.conf", + mode => $::apache::file_mode, content => template('apache/mod/fastcgi.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/fcgid.pp b/apache/manifests/mod/fcgid.pp index 978667033..4c0f91938 100644 --- a/apache/manifests/mod/fcgid.pp +++ b/apache/manifests/mod/fcgid.pp @@ -11,6 +11,7 @@ file { 'unixd_fcgid.conf': ensure => file, path => "${::apache::mod_dir}/unixd_fcgid.conf", + mode => $::apache::file_mode, content => template('apache/mod/unixd_fcgid.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/geoip.pp b/apache/manifests/mod/geoip.pp index 1f8fb08ee..2ff5d2191 100644 --- a/apache/manifests/mod/geoip.pp +++ b/apache/manifests/mod/geoip.pp @@ -22,6 +22,7 @@ file { 'geoip.conf': ensure => file, path => "${::apache::mod_dir}/geoip.conf", + mode => $::apache::file_mode, content => template('apache/mod/geoip.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/info.pp b/apache/manifests/mod/info.pp index f0d03eb0f..bed35af3a 100644 --- a/apache/manifests/mod/info.pp +++ b/apache/manifests/mod/info.pp @@ -10,6 +10,7 @@ file { 'info.conf': ensure => file, path => "${::apache::mod_dir}/info.conf", + mode => $::apache::file_mode, content => template('apache/mod/info.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/itk.pp b/apache/manifests/mod/itk.pp index 990f853ea..2d5bf04c1 100644 --- a/apache/manifests/mod/itk.pp +++ b/apache/manifests/mod/itk.pp @@ -35,7 +35,7 @@ File { owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, } # Template uses: @@ -47,6 +47,7 @@ # - $maxrequestsperchild file { "${::apache::mod_dir}/itk.conf": ensure => file, + mode => $::apache::file_mode, content => template('apache/mod/itk.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], @@ -54,6 +55,26 @@ } case $::osfamily { + 'redhat': { + package { 'httpd-itk': + ensure => present, + } + if versioncmp($apache_version, '2.4') >= 0 { + ::apache::mpm{ 'itk': + apache_version => $apache_version, + } + } + else { + file_line { '/etc/sysconfig/httpd itk enable': + ensure => present, + path => '/etc/sysconfig/httpd', + line => 'HTTPD=/usr/sbin/httpd.itk', + match => '#?HTTPD=/usr/sbin/httpd.itk', + require => Package['httpd'], + notify => Class['apache::service'], + } + } + } 'debian', 'freebsd': { apache::mpm{ 'itk': apache_version => $apache_version, diff --git a/apache/manifests/mod/ldap.pp b/apache/manifests/mod/ldap.pp index d08418671..fe9f6b80a 100644 --- a/apache/manifests/mod/ldap.pp +++ b/apache/manifests/mod/ldap.pp @@ -2,6 +2,11 @@ $apache_version = $::apache::apache_version, $ldap_trusted_global_cert_file = undef, $ldap_trusted_global_cert_type = 'CA_BASE64', + $ldap_shared_cache_size = undef, + $ldap_cache_entries = undef, + $ldap_cache_ttl = undef, + $ldap_opcache_entries = undef, + $ldap_opcache_ttl = undef, ){ if ($ldap_trusted_global_cert_file) { validate_string($ldap_trusted_global_cert_type) @@ -11,6 +16,7 @@ file { 'ldap.conf': ensure => file, path => "${::apache::mod_dir}/ldap.conf", + mode => $::apache::file_mode, content => template('apache/mod/ldap.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/mime.pp b/apache/manifests/mod/mime.pp index ace7663df..0665eb639 100644 --- a/apache/manifests/mod/mime.pp +++ b/apache/manifests/mod/mime.pp @@ -8,6 +8,7 @@ file { 'mime.conf': ensure => file, path => "${::apache::mod_dir}/mime.conf", + mode => $::apache::file_mode, content => template('apache/mod/mime.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/mime_magic.pp b/apache/manifests/mod/mime_magic.pp index c057b01f5..722b0df40 100644 --- a/apache/manifests/mod/mime_magic.pp +++ b/apache/manifests/mod/mime_magic.pp @@ -6,6 +6,7 @@ file { 'mime_magic.conf': ensure => file, path => "${::apache::mod_dir}/mime_magic.conf", + mode => $::apache::file_mode, content => template('apache/mod/mime_magic.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/negotiation.pp b/apache/manifests/mod/negotiation.pp index 02a3a0e64..b9aec3673 100644 --- a/apache/manifests/mod/negotiation.pp +++ b/apache/manifests/mod/negotiation.pp @@ -16,6 +16,7 @@ # Template uses no variables file { 'negotiation.conf': ensure => file, + mode => $::apache::file_mode, path => "${::apache::mod_dir}/negotiation.conf", content => template('apache/mod/negotiation.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], diff --git a/apache/manifests/mod/nss.pp b/apache/manifests/mod/nss.pp index d275cc493..16c285e93 100644 --- a/apache/manifests/mod/nss.pp +++ b/apache/manifests/mod/nss.pp @@ -18,6 +18,7 @@ file { 'nss.conf': ensure => file, path => "${::apache::mod_dir}/nss.conf", + mode => $::apache::file_mode, content => template('apache/mod/nss.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/pagespeed.pp b/apache/manifests/mod/pagespeed.pp index 588849c47..e787d88ef 100644 --- a/apache/manifests/mod/pagespeed.pp +++ b/apache/manifests/mod/pagespeed.pp @@ -47,6 +47,7 @@ file { 'pagespeed.conf': ensure => file, path => "${::apache::mod_dir}/pagespeed.conf", + mode => $::apache::file_mode, content => template('apache/mod/pagespeed.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/passenger.pp b/apache/manifests/mod/passenger.pp index 4f7cb7f04..d0b9f73fe 100644 --- a/apache/manifests/mod/passenger.pp +++ b/apache/manifests/mod/passenger.pp @@ -1,27 +1,29 @@ class apache::mod::passenger ( - $passenger_conf_file = $::apache::params::passenger_conf_file, - $passenger_conf_package_file = $::apache::params::passenger_conf_package_file, - $passenger_high_performance = undef, - $passenger_pool_idle_time = undef, - $passenger_max_requests = undef, - $passenger_spawn_method = undef, - $passenger_stat_throttle_rate = undef, - $rack_autodetect = undef, - $rails_autodetect = undef, - $passenger_root = $::apache::params::passenger_root, - $passenger_ruby = $::apache::params::passenger_ruby, - $passenger_default_ruby = $::apache::params::passenger_default_ruby, - $passenger_max_pool_size = undef, - $passenger_min_instances = undef, - $passenger_use_global_queue = undef, - $passenger_app_env = undef, - $passenger_log_file = undef, - $mod_package = undef, - $mod_package_ensure = undef, - $mod_lib = undef, - $mod_lib_path = undef, - $mod_id = undef, - $mod_path = undef, + $passenger_conf_file = $::apache::params::passenger_conf_file, + $passenger_conf_package_file = $::apache::params::passenger_conf_package_file, + $passenger_high_performance = undef, + $passenger_pool_idle_time = undef, + $passenger_max_request_queue_size = undef, + $passenger_max_requests = undef, + $passenger_spawn_method = undef, + $passenger_stat_throttle_rate = undef, + $rack_autodetect = undef, + $rails_autodetect = undef, + $passenger_root = $::apache::params::passenger_root, + $passenger_ruby = $::apache::params::passenger_ruby, + $passenger_default_ruby = $::apache::params::passenger_default_ruby, + $passenger_max_pool_size = undef, + $passenger_min_instances = undef, + $passenger_use_global_queue = undef, + $passenger_app_env = undef, + $passenger_log_file = undef, + $manage_repo = true, + $mod_package = undef, + $mod_package_ensure = undef, + $mod_lib = undef, + $mod_lib_path = undef, + $mod_id = undef, + $mod_path = undef, ) { if $passenger_spawn_method { @@ -51,6 +53,21 @@ $_lib_path = $mod_lib_path } + if $::osfamily == 'RedHat' and $manage_repo { + yumrepo { 'passenger': + ensure => 'present', + baseurl => 'https://oss-binaries.phusionpassenger.com/yum/passenger/el/$releasever/$basearch', + descr => 'passenger', + enabled => '1', + gpgcheck => '0', + gpgkey => 'https://packagecloud.io/gpg.key', + repo_gpgcheck => '1', + sslcacert => '/etc/pki/tls/certs/ca-bundle.crt', + sslverify => '1', + before => Apache::Mod['passenger'], + } + } + $_id = $mod_id $_path = $mod_path ::apache::mod { 'passenger': diff --git a/apache/manifests/mod/peruser.pp b/apache/manifests/mod/peruser.pp index b6a8015f9..e875a5afc 100644 --- a/apache/manifests/mod/peruser.pp +++ b/apache/manifests/mod/peruser.pp @@ -35,7 +35,7 @@ File { owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, } $mod_dir = $::apache::mod_dir @@ -52,6 +52,7 @@ # - $mod_dir file { "${::apache::mod_dir}/peruser.conf": ensure => file, + mode => $::apache::file_mode, content => template('apache/mod/peruser.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/php.pp b/apache/manifests/mod/php.pp index 1d1274f3b..3d45f87a8 100644 --- a/apache/manifests/mod/php.pp +++ b/apache/manifests/mod/php.pp @@ -50,7 +50,7 @@ path => "${::apache::mod_dir}/php5.conf", owner => 'root', group => $root_group, - mode => '0644', + mode => $::apache::file_mode, content => $manage_content, source => $source, require => [ diff --git a/apache/manifests/mod/prefork.pp b/apache/manifests/mod/prefork.pp index 91567de11..85d8b84d4 100644 --- a/apache/manifests/mod/prefork.pp +++ b/apache/manifests/mod/prefork.pp @@ -24,7 +24,7 @@ File { owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, } # Template uses: diff --git a/apache/manifests/mod/proxy.pp b/apache/manifests/mod/proxy.pp index 8c685d55b..73b054ab3 100644 --- a/apache/manifests/mod/proxy.pp +++ b/apache/manifests/mod/proxy.pp @@ -8,6 +8,7 @@ file { 'proxy.conf': ensure => file, path => "${::apache::mod_dir}/proxy.conf", + mode => $::apache::file_mode, content => template('apache/mod/proxy.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/proxy_html.pp b/apache/manifests/mod/proxy_html.pp index 8b910c251..24f332334 100644 --- a/apache/manifests/mod/proxy_html.pp +++ b/apache/manifests/mod/proxy_html.pp @@ -29,6 +29,7 @@ file { 'proxy_html.conf': ensure => file, path => "${::apache::mod_dir}/proxy_html.conf", + mode => $::apache::file_mode, content => template('apache/mod/proxy_html.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/remoteip.pp b/apache/manifests/mod/remoteip.pp index 564390e94..abceb08c7 100644 --- a/apache/manifests/mod/remoteip.pp +++ b/apache/manifests/mod/remoteip.pp @@ -19,6 +19,7 @@ file { 'remoteip.conf': ensure => file, path => "${::apache::mod_dir}/remoteip.conf", + mode => $::apache::file_mode, content => template('apache/mod/remoteip.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/rpaf.pp b/apache/manifests/mod/rpaf.pp index 12b86eb8b..f21c43ebd 100644 --- a/apache/manifests/mod/rpaf.pp +++ b/apache/manifests/mod/rpaf.pp @@ -12,6 +12,7 @@ file { 'rpaf.conf': ensure => file, path => "${::apache::mod_dir}/rpaf.conf", + mode => $::apache::file_mode, content => template('apache/mod/rpaf.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/security.pp b/apache/manifests/mod/security.pp index 050b1bd6f..95018a680 100644 --- a/apache/manifests/mod/security.pp +++ b/apache/manifests/mod/security.pp @@ -3,6 +3,8 @@ $activated_rules = $::apache::params::modsec_default_rules, $modsec_dir = $::apache::params::modsec_dir, $modsec_secruleengine = $::apache::params::modsec_secruleengine, + $secpcrematchlimit = $::apache::params::secpcrematchlimit, + $secpcrematchlimitrecursion = $::apache::params::secpcrematchlimitrecursion, $allowed_methods = 'GET HEAD POST OPTIONS', $content_types = 'application/x-www-form-urlencoded|multipart/form-data|text/xml|application/xml|application/x-amf', $restricted_extensions = '.asa/ .asax/ .ascx/ .axd/ .backup/ .bak/ .bat/ .cdx/ .cer/ .cfg/ .cmd/ .com/ .config/ .conf/ .cs/ .csproj/ .csr/ .dat/ .db/ .dbf/ .dll/ .dos/ .htr/ .htw/ .ida/ .idc/ .idq/ .inc/ .ini/ .key/ .licx/ .lnk/ .log/ .mdb/ .old/ .pass/ .pdb/ .pol/ .printer/ .pwd/ .resources/ .resx/ .sql/ .sys/ .vb/ .vbs/ .vbproj/ .vsdisco/ .webinfo/ .xsd/ .xsx/', @@ -26,15 +28,18 @@ if $crs_package { package { $crs_package: ensure => 'latest', - before => File['security.conf'], + before => File[$::apache::confd_dir], } } # Template uses: # - $modsec_dir + # - secpcrematchlimit + # - secpcrematchlimitrecursion file { 'security.conf': ensure => file, content => template('apache/mod/security.conf.erb'), + mode => $::apache::file_mode, path => "${::apache::mod_dir}/security.conf", owner => $::apache::params::user, group => $::apache::params::group, diff --git a/apache/manifests/mod/setenvif.pp b/apache/manifests/mod/setenvif.pp index c73102dfb..63d3e321b 100644 --- a/apache/manifests/mod/setenvif.pp +++ b/apache/manifests/mod/setenvif.pp @@ -4,6 +4,7 @@ file { 'setenvif.conf': ensure => file, path => "${::apache::mod_dir}/setenvif.conf", + mode => $::apache::file_mode, content => template('apache/mod/setenvif.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/ssl.pp b/apache/manifests/mod/ssl.pp index a653baded..dcc31ce8f 100644 --- a/apache/manifests/mod/ssl.pp +++ b/apache/manifests/mod/ssl.pp @@ -73,6 +73,7 @@ file { 'ssl.conf': ensure => file, path => "${::apache::mod_dir}/ssl.conf", + mode => $::apache::file_mode, content => template('apache/mod/ssl.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/status.pp b/apache/manifests/mod/status.pp index 4c3f8d9e2..d11a464d7 100644 --- a/apache/manifests/mod/status.pp +++ b/apache/manifests/mod/status.pp @@ -11,7 +11,7 @@ # values are 'On' or 'Off'. Defaults to 'On'. # - $status_path is the path assigned to the Location directive which # defines the URL to access the server status. Defaults to '/server-status'. -# +# # Actions: # - Enable and configure Apache mod_status # @@ -38,6 +38,7 @@ file { 'status.conf': ensure => file, path => "${::apache::mod_dir}/status.conf", + mode => $::apache::file_mode, content => template('apache/mod/status.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/suphp.pp b/apache/manifests/mod/suphp.pp index c50beea06..5d426d794 100644 --- a/apache/manifests/mod/suphp.pp +++ b/apache/manifests/mod/suphp.pp @@ -5,6 +5,7 @@ file {'suphp.conf': ensure => file, path => "${::apache::mod_dir}/suphp.conf", + mode => $::apache::file_mode, content => template('apache/mod/suphp.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/userdir.pp b/apache/manifests/mod/userdir.pp index 4b3d0b8e8..516bb1165 100644 --- a/apache/manifests/mod/userdir.pp +++ b/apache/manifests/mod/userdir.pp @@ -11,6 +11,7 @@ file { 'userdir.conf': ensure => file, path => "${::apache::mod_dir}/userdir.conf", + mode => $::apache::file_mode, content => template('apache/mod/userdir.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/mod/worker.pp b/apache/manifests/mod/worker.pp index 2e6a6421b..9e417e0c4 100644 --- a/apache/manifests/mod/worker.pp +++ b/apache/manifests/mod/worker.pp @@ -79,7 +79,7 @@ File { owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, } # Template uses: diff --git a/apache/manifests/mod/wsgi.pp b/apache/manifests/mod/wsgi.pp index bff5b46b7..d1b821475 100644 --- a/apache/manifests/mod/wsgi.pp +++ b/apache/manifests/mod/wsgi.pp @@ -32,6 +32,7 @@ file {'wsgi.conf': ensure => file, path => "${::apache::mod_dir}/wsgi.conf", + mode => $::apache::file_mode, content => template('apache/mod/wsgi.conf.erb'), require => Exec["mkdir ${::apache::mod_dir}"], before => File[$::apache::mod_dir], diff --git a/apache/manifests/params.pp b/apache/manifests/params.pp index 9e8cad39b..a8cd7bfeb 100644 --- a/apache/manifests/params.pp +++ b/apache/manifests/params.pp @@ -31,20 +31,17 @@ # Default mime types settings $mime_types_additional = { - 'AddHandler' => { - 'type-map' => 'var' - }, - 'AddType' => { - 'text/html' => '.shtml' - }, - 'AddOutputFilter' => { - 'INCLUDES' => '.shtml' - }, + 'AddHandler' => { 'type-map' => 'var', }, + 'AddType' => { 'text/html' => '.shtml', }, + 'AddOutputFilter' => { 'INCLUDES' => '.shtml', }, } # should we use systemd module? $use_systemd = true + # Default mode for files + $file_mode = '0644' + $vhost_include_pattern = '*' if $::operatingsystem == 'Ubuntu' and $::lsbdistrelease == '10.04' { @@ -52,7 +49,7 @@ } else { $verify_command = '/usr/sbin/apachectl -t' } - if $::osfamily == 'RedHat' or $::operatingsystem == 'amazon' { + if $::osfamily == 'RedHat' or $::operatingsystem =~ /^[Aa]mazon$/ { $user = 'apache' $group = 'apache' $root_group = 'root' @@ -62,7 +59,10 @@ $server_root = '/etc/httpd' $conf_dir = "${httpd_dir}/conf" $confd_dir = "${httpd_dir}/conf.d" - $mod_dir = "${httpd_dir}/conf.d" + $mod_dir = $::apache::version::distrelease ? { + '7' => "${httpd_dir}/conf.modules.d", + default => "${httpd_dir}/conf.d", + } $mod_enable_dir = undef $vhost_dir = "${httpd_dir}/conf.d" $vhost_enable_dir = undef @@ -85,9 +85,8 @@ $suphp_addhandler = 'php5-script' $suphp_engine = 'off' $suphp_configpath = undef - # NOTE: The module for Shibboleth is not available to RH/CentOS without an additional repository. http://wiki.aaf.edu.au/tech-info/sp-install-guide - # NOTE: The auth_cas module isn't available to RH/CentOS without enabling EPEL. $mod_packages = { + # NOTE: The auth_cas module isn't available on RH/CentOS without providing dependency packages provided by EPEL. 'auth_cas' => 'mod_auth_cas', 'auth_kerb' => 'mod_auth_kerb', 'auth_mellon' => 'mod_auth_mellon', @@ -103,6 +102,10 @@ default => undef, }, 'pagespeed' => 'mod-pagespeed-stable', + # NOTE: The passenger module isn't available on RH/CentOS without + # providing dependency packages provided by EPEL and passenger + # repositories. See + # https://www.phusionpassenger.com/library/install/apache/install/oss/el7/ 'passenger' => 'mod_passenger', 'perl' => 'mod_perl', 'php5' => $::apache::version::distrelease ? { @@ -112,6 +115,9 @@ 'proxy_html' => 'mod_proxy_html', 'python' => 'mod_python', 'security' => 'mod_security', + # NOTE: The module for Shibboleth is not available on RH/CentOS without + # providing dependency packages provided by Shibboleth's repositories. + # See http://wiki.aaf.edu.au/tech-info/sp-install-guide 'shibboleth' => 'shibboleth', 'ssl' => 'mod_ssl', 'wsgi' => 'mod_wsgi', @@ -153,6 +159,8 @@ $modsec_crs_package = 'mod_security_crs' $modsec_crs_path = '/usr/lib/modsecurity.d' $modsec_dir = '/etc/httpd/modsecurity.d' + $secpcrematchlimit = 1500 + $secpcrematchlimitrecursion = 1500 $modsec_secruleengine = 'On' $modsec_default_rules = [ 'base_rules/modsecurity_35_bad_robots.data', @@ -175,7 +183,7 @@ 'base_rules/modsecurity_crs_49_inbound_blocking.conf', 'base_rules/modsecurity_crs_50_outbound.conf', 'base_rules/modsecurity_crs_59_outbound_blocking.conf', - 'base_rules/modsecurity_crs_60_correlation.conf' + 'base_rules/modsecurity_crs_60_correlation.conf', ] } elsif $::osfamily == 'Debian' { $user = 'www-data' @@ -221,13 +229,20 @@ 'python' => 'libapache2-mod-python', 'rpaf' => 'libapache2-mod-rpaf', 'security' => 'libapache2-modsecurity', + 'shib2' => 'libapache2-mod-shib2', 'suphp' => 'libapache2-mod-suphp', 'wsgi' => 'libapache2-mod-wsgi', 'xsendfile' => 'libapache2-mod-xsendfile', 'shib2' => 'libapache2-mod-shib2', } + if $::osfamily == 'Debian' and versioncmp($::operatingsystemrelease, '8') < 0 { + $shib2_lib = 'mod_shib_22.so' + } else { + $shib2_lib = 'mod_shib2.so' + } $mod_libs = { - 'php5' => 'libphp5.so', + 'php5' => 'libphp5.so', + 'shib2' => $shib2_lib } $conf_template = 'apache/httpd.conf.erb' $keepalive = 'Off' @@ -248,6 +263,8 @@ $modsec_crs_package = 'modsecurity-crs' $modsec_crs_path = '/usr/share/modsecurity-crs' $modsec_dir = '/etc/modsecurity' + $secpcrematchlimit = 1500 + $secpcrematchlimitrecursion = 1500 $modsec_secruleengine = 'On' $modsec_default_rules = [ 'base_rules/modsecurity_35_bad_robots.data', @@ -270,7 +287,7 @@ 'base_rules/modsecurity_crs_49_inbound_blocking.conf', 'base_rules/modsecurity_crs_50_outbound.conf', 'base_rules/modsecurity_crs_59_outbound_blocking.conf', - 'base_rules/modsecurity_crs_60_correlation.conf' + 'base_rules/modsecurity_crs_60_correlation.conf', ] $alias_icons_path = '/usr/share/apache2/icons' $error_documents_path = '/usr/share/apache2/error' @@ -432,19 +449,20 @@ $suphp_configpath = '/etc/php5/apache2' $mod_packages = { # NOTE: I list here only modules that are not included in www-servers/apache - 'auth_kerb' => 'www-apache/mod_auth_kerb', - 'fcgid' => 'www-apache/mod_fcgid', - 'passenger' => 'www-apache/passenger', - 'perl' => 'www-apache/mod_perl', - 'php5' => 'dev-lang/php', - 'proxy_html' => 'www-apache/mod_proxy_html', - 'proxy_fcgi' => 'www-apache/mod_proxy_fcgi', - 'python' => 'www-apache/mod_python', - 'wsgi' => 'www-apache/mod_wsgi', - 'dav_svn' => 'dev-vcs/subversion', - 'xsendfile' => 'www-apache/mod_xsendfile', - 'rpaf' => 'www-apache/mod_rpaf', - 'xml2enc' => 'www-apache/mod_xml2enc', + 'auth_kerb' => 'www-apache/mod_auth_kerb', + 'authnz_external' => 'www-apache/mod_authnz_external', + 'fcgid' => 'www-apache/mod_fcgid', + 'passenger' => 'www-apache/passenger', + 'perl' => 'www-apache/mod_perl', + 'php5' => 'dev-lang/php', + 'proxy_html' => 'www-apache/mod_proxy_html', + 'proxy_fcgi' => 'www-apache/mod_proxy_fcgi', + 'python' => 'www-apache/mod_python', + 'wsgi' => 'www-apache/mod_wsgi', + 'dav_svn' => 'dev-vcs/subversion', + 'xsendfile' => 'www-apache/mod_xsendfile', + 'rpaf' => 'www-apache/mod_rpaf', + 'xml2enc' => 'www-apache/mod_xml2enc', } $mod_libs = { 'php5' => 'libphp5.so', diff --git a/apache/manifests/vhost.pp b/apache/manifests/vhost.pp index 7e48317a8..912be76e5 100644 --- a/apache/manifests/vhost.pp +++ b/apache/manifests/vhost.pp @@ -25,6 +25,7 @@ $ssl_honorcipherorder = undef, $ssl_verify_client = undef, $ssl_verify_depth = undef, + $ssl_proxy_verify = undef, $ssl_proxy_check_peer_cn = undef, $ssl_proxy_check_peer_name = undef, $ssl_proxy_machine_cert = undef, @@ -158,7 +159,9 @@ validate_bool($ssl_proxyengine) if $rewrites { validate_array($rewrites) - validate_hash($rewrites[0]) + unless empty($rewrites) { + validate_hash($rewrites[0]) + } } # Input validation begins @@ -234,6 +237,10 @@ validate_string($docroot) } + if $ssl_proxy_verify { + validate_re($ssl_proxy_verify,'^(none|optional|require|optional_no_ca)$',"${ssl_proxy_verify} is not permitted for ssl_proxy_verify. Allowed values are 'none', 'optional', 'require' or 'optional_no_ca'.") + } + if $ssl_proxy_check_peer_cn { validate_re($ssl_proxy_check_peer_cn,'(^on$|^off$)',"${ssl_proxy_check_peer_cn} is not permitted for ssl_proxy_check_peer_cn. Allowed values are 'on' or 'off'.") } @@ -497,7 +504,7 @@ path => "${::apache::vhost_dir}/${priority_real}${filename}.conf", owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, order => 'numeric', require => Package['httpd'], notify => Class['apache::service'], @@ -516,7 +523,7 @@ target => "${::apache::vhost_dir}/${priority_real}${filename}.conf", owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, require => Concat["${priority_real}${filename}.conf"], notify => Class['apache::service'], } @@ -674,6 +681,26 @@ } } + # Template uses: + # - $headers + if $headers and ! empty($headers) { + concat::fragment { "${name}-header": + target => "${priority_real}${filename}.conf", + order => 140, + content => template('apache/vhost/_header.erb'), + } + } + + # Template uses: + # - $request_headers + if $request_headers and ! empty($request_headers) { + concat::fragment { "${name}-requestheader": + target => "${priority_real}${filename}.conf", + order => 150, + content => template('apache/vhost/_requestheader.erb'), + } + } + # Template uses: # - $proxy_dest # - $proxy_pass @@ -683,7 +710,7 @@ if $proxy_dest or $proxy_pass or $proxy_pass_match or $proxy_dest_match { concat::fragment { "${name}-proxy": target => "${priority_real}${filename}.conf", - order => 140, + order => 160, content => template('apache/vhost/_proxy.erb'), } } @@ -693,7 +720,7 @@ if $rack_base_uris { concat::fragment { "${name}-rack": target => "${priority_real}${filename}.conf", - order => 150, + order => 170, content => template('apache/vhost/_rack.erb'), } } @@ -703,7 +730,7 @@ if $passenger_base_uris { concat::fragment { "${name}-passenger_uris": target => "${priority_real}${filename}.conf", - order => 155, + order => 175, content => template('apache/vhost/_passenger_base_uris.erb'), } } @@ -721,10 +748,10 @@ # - $redirectmatch_status_a # - $redirectmatch_regexp_a # - $redirectmatch_dest - if ($redirect_source and $redirect_dest) or ($redirectmatch_status and $redirectmatch_regexp and $redirectmatch_dest) { + if ($redirect_source and $redirect_dest) or ($redirectmatch_regexp and $redirectmatch_dest) { concat::fragment { "${name}-redirect": target => "${priority_real}${filename}.conf", - order => 160, + order => 180, content => template('apache/vhost/_redirect.erb'), } } @@ -738,7 +765,7 @@ if $rewrites or $rewrite_rule { concat::fragment { "${name}-rewrite": target => "${priority_real}${filename}.conf", - order => 170, + order => 190, content => template('apache/vhost/_rewrite.erb'), } } @@ -749,7 +776,7 @@ if ( $scriptalias or $scriptaliases != [] ) { concat::fragment { "${name}-scriptalias": target => "${priority_real}${filename}.conf", - order => 180, + order => 200, content => template('apache/vhost/_scriptalias.erb'), } } @@ -759,7 +786,7 @@ if $serveraliases and ! empty($serveraliases) { concat::fragment { "${name}-serveralias": target => "${priority_real}${filename}.conf", - order => 190, + order => 210, content => template('apache/vhost/_serveralias.erb'), } } @@ -770,7 +797,7 @@ if ($setenv and ! empty($setenv)) or ($setenvif and ! empty($setenvif)) { concat::fragment { "${name}-setenv": target => "${priority_real}${filename}.conf", - order => 200, + order => 220, content => template('apache/vhost/_setenv.erb'), } } @@ -796,20 +823,21 @@ if $ssl { concat::fragment { "${name}-ssl": target => "${priority_real}${filename}.conf", - order => 210, + order => 230, content => template('apache/vhost/_ssl.erb'), } } # Template uses: # - $ssl_proxyengine + # - $ssl_proxy_verify # - $ssl_proxy_check_peer_cn # - $ssl_proxy_check_peer_name # - $ssl_proxy_machine_cert if $ssl_proxyengine { concat::fragment { "${name}-sslproxy": target => "${priority_real}${filename}.conf", - order => 210, + order => 230, content => template('apache/vhost/_sslproxy.erb'), } } @@ -825,7 +853,7 @@ if $auth_kerb { concat::fragment { "${name}-auth_kerb": target => "${priority_real}${filename}.conf", - order => 210, + order => 230, content => template('apache/vhost/_auth_kerb.erb'), } } @@ -837,7 +865,7 @@ if $suphp_engine == 'on' { concat::fragment { "${name}-suphp": target => "${priority_real}${filename}.conf", - order => 220, + order => 240, content => template('apache/vhost/_suphp.erb'), } } @@ -848,7 +876,7 @@ if ($php_values and ! empty($php_values)) or ($php_flags and ! empty($php_flags)) { concat::fragment { "${name}-php": target => "${priority_real}${filename}.conf", - order => 220, + order => 240, content => template('apache/vhost/_php.erb'), } } @@ -858,29 +886,9 @@ # - $php_admin_flags if ($php_admin_values and ! empty($php_admin_values)) or ($php_admin_flags and ! empty($php_admin_flags)) { concat::fragment { "${name}-php_admin": - target => "${priority_real}${filename}.conf", - order => 230, - content => template('apache/vhost/_php_admin.erb'), - } - } - - # Template uses: - # - $headers - if $headers and ! empty($headers) { - concat::fragment { "${name}-header": - target => "${priority_real}${filename}.conf", - order => 240, - content => template('apache/vhost/_header.erb'), - } - } - - # Template uses: - # - $request_headers - if $request_headers and ! empty($request_headers) { - concat::fragment { "${name}-requestheader": target => "${priority_real}${filename}.conf", order => 250, - content => template('apache/vhost/_requestheader.erb'), + content => template('apache/vhost/_php_admin.erb'), } } diff --git a/apache/manifests/vhost/custom.pp b/apache/manifests/vhost/custom.pp index d85e4d091..12567f5db 100644 --- a/apache/manifests/vhost/custom.pp +++ b/apache/manifests/vhost/custom.pp @@ -30,7 +30,7 @@ target => "${::apache::vhost_dir}/${priority}-${filename}.conf", owner => 'root', group => $::apache::params::root_group, - mode => '0644', + mode => $::apache::file_mode, require => Apache::Custom_config[$filename], } } diff --git a/apache/spec/acceptance/apache_parameters_spec.rb b/apache/spec/acceptance/apache_parameters_spec.rb index 00e486fb6..923df6666 100644 --- a/apache/spec/acceptance/apache_parameters_spec.rb +++ b/apache/spec/acceptance/apache_parameters_spec.rb @@ -55,7 +55,11 @@ class { 'apache': describe service($service_name) do it { is_expected.to be_running } - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { is_expected.to be_enabled } + end end end @@ -72,7 +76,11 @@ class { 'apache': describe service($service_name) do it { is_expected.not_to be_running } - it { is_expected.not_to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { is_expected.not_to be_enabled } + end end end @@ -90,7 +98,11 @@ class { 'apache': describe service($service_name) do it { is_expected.not_to be_running } - it { is_expected.not_to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { is_expected.not_to be_enabled } + end end end diff --git a/apache/spec/acceptance/apache_ssl_spec.rb b/apache/spec/acceptance/apache_ssl_spec.rb index ccf65c727..254a3c35a 100644 --- a/apache/spec/acceptance/apache_ssl_spec.rb +++ b/apache/spec/acceptance/apache_ssl_spec.rb @@ -1,13 +1,6 @@ require 'spec_helper_acceptance' require_relative './version.rb' -case fact('osfamily') -when 'RedHat' - vhostd = '/etc/httpd/conf.d' -when 'Debian' - vhostd = '/etc/apache2/sites-available' -end - describe 'apache ssl' do describe 'ssl parameters' do @@ -28,7 +21,7 @@ class { 'apache': apply_manifest(pp, :catch_failures => true) end - describe file("#{vhostd}/15-default-ssl.conf") do + describe file("#{$vhost_dir}/15-default-ssl.conf") do it { is_expected.to be_file } it { is_expected.to contain 'SSLCertificateFile "/tmp/ssl_cert"' } it { is_expected.to contain 'SSLCertificateKeyFile "/tmp/ssl_key"' } @@ -74,7 +67,7 @@ class { 'apache': apply_manifest(pp, :catch_failures => true) end - describe file("#{vhostd}/25-test_ssl.conf") do + describe file("#{$vhost_dir}/25-test_ssl.conf") do it { is_expected.to be_file } it { is_expected.to contain 'SSLCertificateFile "/tmp/ssl_cert"' } it { is_expected.to contain 'SSLCertificateKeyFile "/tmp/ssl_key"' } diff --git a/apache/spec/acceptance/class_spec.rb b/apache/spec/acceptance/class_spec.rb index 47b0d36fa..aff79eb08 100644 --- a/apache/spec/acceptance/class_spec.rb +++ b/apache/spec/acceptance/class_spec.rb @@ -1,38 +1,26 @@ require 'spec_helper_acceptance' +require_relative './version.rb' describe 'apache class' do - case fact('osfamily') - when 'RedHat' - package_name = 'httpd' - service_name = 'httpd' - when 'Debian' - package_name = 'apache2' - service_name = 'apache2' - when 'FreeBSD' - package_name = 'apache24' - service_name = 'apache24' - when 'Gentoo' - package_name = 'www-servers/apache' - service_name = 'apache2' - end - context 'default parameters' do - it 'should work with no errors' do - pp = <<-EOS - class { 'apache': } + let(:pp) do + <<-EOS + class { 'apache': } EOS - - # Run it twice and test for idempotency - apply_manifest(pp, :catch_failures => true) - expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero end + # Run it twice and test for idempotency + it_behaves_like "a idempotent resource" - describe package(package_name) do + describe package($package_name) do it { is_expected.to be_installed } end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -43,45 +31,48 @@ class { 'apache': } context 'custom site/mod dir parameters' do # Using puppet_apply as a helper - it 'should work with no errors' do - pp = <<-EOS - if $::osfamily == 'RedHat' and "$::selinux" == "true" { - $semanage_package = $::operatingsystemmajrelease ? { - '5' => 'policycoreutils', - default => 'policycoreutils-python', - } + let(:pp) do + <<-EOS + if $::osfamily == 'RedHat' and "$::selinux" == "true" { + $semanage_package = $::operatingsystemmajrelease ? { + '5' => 'policycoreutils', + default => 'policycoreutils-python', + } - package { $semanage_package: ensure => installed } - exec { 'set_apache_defaults': - command => 'semanage fcontext -a -t httpd_sys_content_t "/apache_spec(/.*)?"', - path => '/bin:/usr/bin/:/sbin:/usr/sbin', - subscribe => Package[$semanage_package], - refreshonly => true, + package { $semanage_package: ensure => installed } + exec { 'set_apache_defaults': + command => 'semanage fcontext -a -t httpd_sys_content_t "/apache_spec(/.*)?"', + path => '/bin:/usr/bin/:/sbin:/usr/sbin', + subscribe => Package[$semanage_package], + refreshonly => true, + } + exec { 'restorecon_apache': + command => 'restorecon -Rv /apache_spec', + path => '/bin:/usr/bin/:/sbin:/usr/sbin', + before => Service['httpd'], + require => Class['apache'], + subscribe => Exec['set_apache_defaults'], + refreshonly => true, + } } - exec { 'restorecon_apache': - command => 'restorecon -Rv /apache_spec', - path => '/bin:/usr/bin/:/sbin:/usr/sbin', - before => Service['httpd'], - require => Class['apache'], - subscribe => Exec['set_apache_defaults'], - refreshonly => true, + file { '/apache_spec': ensure => directory, } + file { '/apache_spec/apache_custom': ensure => directory, } + class { 'apache': + mod_dir => '/apache_spec/apache_custom/mods', + vhost_dir => '/apache_spec/apache_custom/vhosts', } - } - file { '/apache_spec': ensure => directory, } - file { '/apache_spec/apache_custom': ensure => directory, } - class { 'apache': - mod_dir => '/apache_spec/apache_custom/mods', - vhost_dir => '/apache_spec/apache_custom/vhosts', - } EOS - - # Run it twice and test for idempotency - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + # Run it twice and test for idempotency + it_behaves_like "a idempotent resource" + + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end end diff --git a/apache/spec/acceptance/default_mods_spec.rb b/apache/spec/acceptance/default_mods_spec.rb index c2d6a8c26..3f2852696 100644 --- a/apache/spec/acceptance/default_mods_spec.rb +++ b/apache/spec/acceptance/default_mods_spec.rb @@ -1,36 +1,20 @@ require 'spec_helper_acceptance' - -case fact('osfamily') -when 'RedHat' - mod_dir = '/etc/httpd/conf.d' - servicename = 'httpd' -when 'Debian' - mod_dir = '/etc/apache2/mods-available' - servicename = 'apache2' -when 'FreeBSD' - mod_dir = '/usr/local/etc/apache24/Modules' - servicename = 'apache24' -when 'Gentoo' - mod_dir = '/etc/apache2/modules.d' - servicename = 'apache2' -end +require_relative './version.rb' describe 'apache::default_mods class' do describe 'no default mods' do # Using puppet_apply as a helper - it 'should apply with no errors' do - pp = <<-EOS + let(:pp) do + <<-EOS class { 'apache': default_mods => false, } EOS - - # Run it twice and test for idempotency - apply_manifest(pp, :catch_failures => true) - expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero end - describe service(servicename) do + # Run it twice and test for idempotency + it_behaves_like "a idempotent resource" + describe service($service_name) do it { is_expected.to be_running } end end @@ -56,20 +40,20 @@ class { 'apache': end # Are these the same? - describe service(servicename) do + describe service($service_name) do it { is_expected.not_to be_running } end - describe "service #{servicename}" do + describe "service #{$service_name}" do it 'should not be running' do - shell("pidof #{servicename}", {:acceptable_exit_codes => 1}) + shell("pidof #{$service_name}", {:acceptable_exit_codes => 1}) end end end describe 'alternative default mods' do # Using puppet_apply as a helper - it 'should apply with no errors' do - pp = <<-EOS + let(:pp) do + <<-EOS class { 'apache': default_mods => [ 'info', @@ -88,35 +72,30 @@ class { 'apache': setenv => 'TEST1 one', } EOS - - apply_manifest(pp, :catch_failures => true) - shell('sleep 10') - expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero end + it_behaves_like "a idempotent resource" - describe service(servicename) do + describe service($service_name) do it { is_expected.to be_running } end end describe 'change loadfile name' do - it 'should apply with no errors' do - pp = <<-EOS + let(:pp) do + <<-EOS class { 'apache': default_mods => false } ::apache::mod { 'auth_basic': loadfile_name => 'zz_auth_basic.load', } EOS - # Run it twice and test for idempotency - apply_manifest(pp, :catch_failures => true) - expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero end - - describe service(servicename) do + # Run it twice and test for idempotency + it_behaves_like "a idempotent resource" + describe service($service_name) do it { is_expected.to be_running } end - describe file("#{mod_dir}/zz_auth_basic.load") do + describe file("#{$mod_dir}/zz_auth_basic.load") do it { is_expected.to be_file } end end diff --git a/apache/spec/acceptance/itk_spec.rb b/apache/spec/acceptance/itk_spec.rb index c220a70f8..059589a3f 100644 --- a/apache/spec/acceptance/itk_spec.rb +++ b/apache/spec/acceptance/itk_spec.rb @@ -4,11 +4,21 @@ when 'Debian' service_name = 'apache2' majrelease = fact('operatingsystemmajrelease') - if [ '6', '7', '10.04', '12.04'].include?(majrelease) + if ['6', '7', '10.04', '12.04'].include?(majrelease) variant = :itk_only else variant = :prefork end +when 'RedHat' + unless fact('operatingsystemmajrelease') == '5' + service_name = 'httpd' + majrelease = fact('operatingsystemmajrelease') + if ['6'].include?(majrelease) + variant = :itk_only + else + variant = :prefork + end + end when 'FreeBSD' service_name = 'apache24' majrelease = fact('operatingsystemmajrelease') @@ -18,31 +28,33 @@ describe 'apache::mod::itk class', :if => service_name do describe 'running puppet code' do # Using puppet_apply as a helper - it 'should work with no errors' do - pp = case variant - when :prefork - <<-EOS - class { 'apache': - mpm_module => 'prefork', - } - class { 'apache::mod::itk': } - EOS - when :itk_only - <<-EOS - class { 'apache': - mpm_module => 'itk', - } - EOS - end - - # Run it twice and test for idempotency - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) + let(:pp) do + case variant + when :prefork + <<-EOS + class { 'apache': + mpm_module => 'prefork', + } + class { 'apache::mod::itk': } + EOS + when :itk_only + <<-EOS + class { 'apache': + mpm_module => 'itk', + } + EOS + end end + # Run it twice and test for idempotency + it_behaves_like "a idempotent resource" end describe service(service_name) do it { is_expected.to be_running } - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end end end diff --git a/apache/spec/acceptance/mod_dav_svn_spec.rb b/apache/spec/acceptance/mod_dav_svn_spec.rb index 10c9b77d7..9abd12aed 100644 --- a/apache/spec/acceptance/mod_dav_svn_spec.rb +++ b/apache/spec/acceptance/mod_dav_svn_spec.rb @@ -1,22 +1,17 @@ require 'spec_helper_acceptance' +require_relative './version.rb' describe 'apache::mod::dav_svn class', :unless => (fact('operatingsystem') == 'OracleLinux' and fact('operatingsystemmajrelease') == '7') do case fact('osfamily') when 'Debian' - mod_dir = '/etc/apache2/mods-available' - service_name = 'apache2' if fact('operatingsystemmajrelease') == '6' or fact('operatingsystemmajrelease') == '10.04' or fact('operatingsystemrelease') == '10.04' authz_svn_load_file = 'dav_svn_authz_svn.load' else authz_svn_load_file = 'authz_svn.load' end when 'RedHat' - mod_dir = '/etc/httpd/conf.d' - service_name = 'httpd' authz_svn_load_file = 'dav_svn_authz_svn.load' when 'FreeBSD' - mod_dir = '/usr/local/etc/apache24/Modules' - service_name = 'apache24' authz_svn_load_file = 'dav_svn_authz_svn.load' end @@ -29,12 +24,16 @@ class { 'apache': } apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end - describe file("#{mod_dir}/dav_svn.load") do + describe file("#{$mod_dir}/dav_svn.load") do it { is_expected.to contain "LoadModule dav_svn_module" } end end @@ -50,12 +49,16 @@ class { 'apache::mod::dav_svn': apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end - describe file("#{mod_dir}/#{authz_svn_load_file}") do + describe file("#{$mod_dir}/#{authz_svn_load_file}") do it { is_expected.to contain "LoadModule authz_svn_module" } end end diff --git a/apache/spec/acceptance/mod_deflate_spec.rb b/apache/spec/acceptance/mod_deflate_spec.rb index 3b505bdbf..1b55e087a 100644 --- a/apache/spec/acceptance/mod_deflate_spec.rb +++ b/apache/spec/acceptance/mod_deflate_spec.rb @@ -1,21 +1,7 @@ require 'spec_helper_acceptance' +require_relative './version.rb' describe 'apache::mod::deflate class' do - case fact('osfamily') - when 'Debian' - mod_dir = '/etc/apache2/mods-available' - service_name = 'apache2' - when 'RedHat' - mod_dir = '/etc/httpd/conf.d' - service_name = 'httpd' - when 'FreeBSD' - mod_dir = '/usr/local/etc/apache24/Modules' - service_name = 'apache24' - when 'Gentoo' - mod_dir = '/etc/apache2/modules.d' - service_name = 'apache2' - end - context "default deflate config" do it 'succeeds in puppeting deflate' do pp= <<-EOS @@ -25,12 +11,16 @@ class { 'apache': } apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end - describe file("#{mod_dir}/deflate.conf") do + describe file("#{$mod_dir}/deflate.conf") do it { is_expected.to contain "AddOutputFilterByType DEFLATE text/html text/plain text/xml" } it { is_expected.to contain "AddOutputFilterByType DEFLATE text/css" } it { is_expected.to contain "AddOutputFilterByType DEFLATE application/x-javascript application/javascript application/ecmascript" } diff --git a/apache/spec/acceptance/mod_mime_spec.rb b/apache/spec/acceptance/mod_mime_spec.rb index e47360b5e..f8bc7c15c 100644 --- a/apache/spec/acceptance/mod_mime_spec.rb +++ b/apache/spec/acceptance/mod_mime_spec.rb @@ -1,21 +1,7 @@ require 'spec_helper_acceptance' +require_relative './version.rb' describe 'apache::mod::mime class' do - case fact('osfamily') - when 'Debian' - mod_dir = '/etc/apache2/mods-available' - service_name = 'apache2' - when 'RedHat' - mod_dir = '/etc/httpd/conf.d' - service_name = 'httpd' - when 'FreeBSD' - mod_dir = '/usr/local/etc/apache24/Modules' - service_name = 'apache24' - when 'Gentoo' - mod_dir = '/etc/apache2/modules.d' - service_name = 'apache2' - end - context "default mime config" do it 'succeeds in puppeting mime' do pp= <<-EOS @@ -25,12 +11,16 @@ class { 'apache': } apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end - describe file("#{mod_dir}/mime.conf") do + describe file("#{$mod_dir}/mime.conf") do it { is_expected.to contain "AddType application/x-compress .Z" } it { is_expected.to contain "AddHandler type-map var\n" } it { is_expected.to contain "AddType text/html .shtml\n" } diff --git a/apache/spec/acceptance/mod_negotiation_spec.rb b/apache/spec/acceptance/mod_negotiation_spec.rb index 48eb896b6..56c29e318 100644 --- a/apache/spec/acceptance/mod_negotiation_spec.rb +++ b/apache/spec/acceptance/mod_negotiation_spec.rb @@ -1,25 +1,7 @@ require 'spec_helper_acceptance' +require_relative './version.rb' describe 'apache::mod::negotiation class' do - case fact('osfamily') - when 'Debian' - vhost_dir = '/etc/apache2/sites-enabled' - mod_dir = '/etc/apache2/mods-available' - service_name = 'apache2' - when 'RedHat' - vhost_dir = '/etc/httpd/conf.d' - mod_dir = '/etc/httpd/conf.d' - service_name = 'httpd' - when 'FreeBSD' - vhost_dir = '/usr/local/etc/apache24/Vhosts' - mod_dir = '/usr/local/etc/apache24/Modules' - service_name = 'apache24' - when 'Gentoo' - vhost_dir = '/etc/apache2/vhosts.d' - mod_dir = '/etc/apache2/modules.d' - service_name = 'apache2' - end - context "default negotiation config" do it 'succeeds in puppeting negotiation' do pp= <<-EOS @@ -29,13 +11,17 @@ class { '::apache::mod::negotiation': } apply_manifest(pp, :catch_failures => true) end - describe file("#{mod_dir}/negotiation.conf") do + describe file("#{$mod_dir}/negotiation.conf") do it { should contain "LanguagePriority en ca cs da de el eo es et fr he hr it ja ko ltz nl nn no pl pt pt-BR ru sv zh-CN zh-TW ForceLanguagePriority Prefer Fallback" } end - describe service(service_name) do - it { should be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { should be_running } end end @@ -51,12 +37,16 @@ class { '::apache::mod::negotiation': apply_manifest(pp, :catch_failures => true) end - describe file("#{mod_dir}/negotiation.conf") do + describe file("#{$mod_dir}/negotiation.conf") do it { should contain "ForceLanguagePriority Prefer" } end - describe service(service_name) do - it { should be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { should be_running } end end @@ -72,12 +62,16 @@ class { '::apache::mod::negotiation': apply_manifest(pp, :catch_failures => true) end - describe file("#{mod_dir}/negotiation.conf") do + describe file("#{$mod_dir}/negotiation.conf") do it { should contain "LanguagePriority en es" } end - describe service(service_name) do - it { should be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { should be_running } end end diff --git a/apache/spec/acceptance/mod_pagespeed_spec.rb b/apache/spec/acceptance/mod_pagespeed_spec.rb index f8060a167..009df6a10 100644 --- a/apache/spec/acceptance/mod_pagespeed_spec.rb +++ b/apache/spec/acceptance/mod_pagespeed_spec.rb @@ -1,25 +1,7 @@ require 'spec_helper_acceptance' +require_relative './version.rb' describe 'apache::mod::pagespeed class' do - case fact('osfamily') - when 'Debian' - vhost_dir = '/etc/apache2/sites-enabled' - mod_dir = '/etc/apache2/mods-available' - service_name = 'apache2' - when 'RedHat' - vhost_dir = '/etc/httpd/conf.d' - mod_dir = '/etc/httpd/conf.d' - service_name = 'httpd' - when 'FreeBSD' - vhost_dir = '/usr/local/etc/apache24/Vhosts' - mod_dir = '/usr/local/etc/apache24/Modules' - service_name = 'apache24' - when 'Gentoo' - vhost_dir = '/etc/apache2/vhosts.d' - mod_dir = '/etc/apache2/modules.d' - service_name = 'apache2' - end - context "default pagespeed config" do it 'succeeds in puppeting pagespeed' do pp= <<-EOS @@ -66,12 +48,16 @@ class { 'apache::mod::pagespeed': apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end - describe file("#{mod_dir}/pagespeed.conf") do + describe file("#{$mod_dir}/pagespeed.conf") do it { is_expected.to contain "AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER text/html" } it { is_expected.to contain "ModPagespeedEnableFilters remove_comments" } it { is_expected.to contain "ModPagespeedDisableFilters extend_cache" } diff --git a/apache/spec/acceptance/mod_passenger_spec.rb b/apache/spec/acceptance/mod_passenger_spec.rb index df9cd9e13..fd967d1ea 100644 --- a/apache/spec/acceptance/mod_passenger_spec.rb +++ b/apache/spec/acceptance/mod_passenger_spec.rb @@ -1,12 +1,12 @@ require 'spec_helper_acceptance' +require_relative './version.rb' describe 'apache::mod::passenger class' do case fact('osfamily') when 'Debian' - service_name = 'apache2' mod_dir = '/etc/apache2/mods-available/' - conf_file = "#{mod_dir}passenger.conf" - load_file = "#{mod_dir}zpassenger.load" + conf_file = "#{$mod_dir}/passenger.conf" + load_file = "#{$mod_dir}/zpassenger.load" case fact('operatingsystem') when 'Ubuntu' @@ -46,43 +46,37 @@ rackapp_user = 'www-data' rackapp_group = 'www-data' when 'RedHat' - service_name = 'httpd' - mod_dir = '/etc/httpd/conf.d/' - conf_file = "#{mod_dir}passenger.conf" - load_file = "#{mod_dir}zpassenger.load" + conf_file = "#{$mod_dir}/passenger.conf" + load_file = "#{$mod_dir}/zpassenger.load" # sometimes installs as 3.0.12, sometimes as 3.0.19 - so just check for the stable part - passenger_root = '/usr/lib/ruby/gems/1.8/gems/passenger-3.0.1' + passenger_root = '/usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini' passenger_ruby = '/usr/bin/ruby' - passenger_tempdir = '/var/run/rubygem-passenger' passenger_module_path = 'modules/mod_passenger.so' rackapp_user = 'apache' rackapp_group = 'apache' end pp_rackapp = <<-EOS - /* a simple ruby rack 'hellow world' app */ - file { '/var/www/passenger': - ensure => directory, - owner => '#{rackapp_user}', - group => '#{rackapp_group}', - require => Class['apache::mod::passenger'], - } - file { '/var/www/passenger/config.ru': - ensure => file, - owner => '#{rackapp_user}', - group => '#{rackapp_group}', - content => "app = proc { |env| [200, { \\"Content-Type\\" => \\"text/html\\" }, [\\"hello world\\"]] }\\nrun app", - require => File['/var/www/passenger'] , - } - apache::vhost { 'passenger.example.com': - port => '80', - docroot => '/var/www/passenger/public', - docroot_group => '#{rackapp_group}' , - docroot_owner => '#{rackapp_user}' , - custom_fragment => "PassengerRuby #{passenger_ruby}\\nRailsEnv development" , - require => File['/var/www/passenger/config.ru'] , - } - host { 'passenger.example.com': ip => '127.0.0.1', } + /* a simple ruby rack 'hello world' app */ + file { '/var/www/passenger': + ensure => directory, + owner => '#{rackapp_user}', + group => '#{rackapp_group}', + } + file { '/var/www/passenger/config.ru': + ensure => file, + owner => '#{rackapp_user}', + group => '#{rackapp_group}', + content => "app = proc { |env| [200, { \\"Content-Type\\" => \\"text/html\\" }, [\\"hello world\\"]] }\\nrun app", + } + apache::vhost { 'passenger.example.com': + port => '80', + docroot => '/var/www/passenger/public', + docroot_group => '#{rackapp_group}', + docroot_owner => '#{rackapp_user}', + require => File['/var/www/passenger/config.ru'], + } + host { 'passenger.example.com': ip => '127.0.0.1', } EOS case fact('osfamily') @@ -98,8 +92,12 @@ class { 'apache::mod::passenger': } apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -152,7 +150,7 @@ class { 'apache::mod::passenger': } # passenger-memory-stats output on newer Debian/Ubuntu verions do not contain # these two lines unless ((fact('operatingsystem') == 'Ubuntu' && fact('operatingsystemrelease') == '14.04') or - (fact('operatingsystem') == 'Debian' && fact('operatingsystemrelease') == '8.0')) + (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8')) expect(r.stdout).to match(/### Processes: [0-9]+/) expect(r.stdout).to match(/### Total private dirty RSS: [0-9\.]+ MB/) end @@ -169,7 +167,8 @@ class { 'apache::mod::passenger': } shell("PATH=/usr/bin:$PATH /usr/sbin/passenger-status") do |r| # spacing may vary expect(r.stdout).to match(/[\-]+ General information [\-]+/) - if fact('operatingsystem') == 'Ubuntu' && fact('operatingsystemrelease') == '14.04' + if fact('operatingsystem') == 'Ubuntu' && fact('operatingsystemrelease') == '14.04' or + fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8' expect(r.stdout).to match(/Max pool size[ ]+: [0-9]+/) expect(r.stdout).to match(/Processes[ ]+: [0-9]+/) expect(r.stdout).to match(/Requests in top-level queue[ ]+: [0-9]+/) diff --git a/apache/spec/acceptance/mod_php_spec.rb b/apache/spec/acceptance/mod_php_spec.rb index a42f52373..11bcafcba 100644 --- a/apache/spec/acceptance/mod_php_spec.rb +++ b/apache/spec/acceptance/mod_php_spec.rb @@ -1,25 +1,7 @@ require 'spec_helper_acceptance' +require_relative './version.rb' describe 'apache::mod::php class' do - case fact('osfamily') - when 'Debian' - vhost_dir = '/etc/apache2/sites-enabled' - mod_dir = '/etc/apache2/mods-available' - service_name = 'apache2' - when 'RedHat' - vhost_dir = '/etc/httpd/conf.d' - mod_dir = '/etc/httpd/conf.d' - service_name = 'httpd' - when 'FreeBSD' - vhost_dir = '/usr/local/etc/apache24/Vhosts' - mod_dir = '/usr/local/etc/apache24/Modules' - service_name = 'apache24' - when 'Gentoo' - vhost_dir = '/etc/apache2/vhosts.d' - mod_dir = '/etc/apache2/modules.d' - service_name = 'apache2' - end - context "default php config" do it 'succeeds in puppeting php' do pp= <<-EOS @@ -40,12 +22,16 @@ class { 'apache::mod::php': } apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end - describe file("#{mod_dir}/php5.conf") do + describe file("#{$mod_dir}/php5.conf") do it { is_expected.to contain "DirectoryIndex index.php" } end @@ -83,12 +69,16 @@ class { 'apache::mod::php': apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end - describe file("#{vhost_dir}/25-php.example.com.conf") do + describe file("#{$vhost_dir}/25-php.example.com.conf") do it { is_expected.to contain " php_flag display_errors on" } it { is_expected.to contain " php_value include_path .:/usr/share/pear:/usr/bin/php" } it { is_expected.to contain " php_admin_flag engine on" } @@ -116,7 +106,7 @@ class {'apache::mod::php': apply_manifest(pp, :catch_failures => true) end - describe file("#{mod_dir}/php5.conf") do + describe file("#{$mod_dir}/php5.conf") do it { should contain "# somecontent" } end end @@ -135,7 +125,7 @@ class {'apache::mod::php': apply_manifest(pp, :catch_failures => true) end - describe file("#{mod_dir}/php5.conf") do + describe file("#{$mod_dir}/php5.conf") do it { should contain "# somecontent" } end end diff --git a/apache/spec/acceptance/mod_proxy_html_spec.rb b/apache/spec/acceptance/mod_proxy_html_spec.rb index 840ea563f..f87d82583 100644 --- a/apache/spec/acceptance/mod_proxy_html_spec.rb +++ b/apache/spec/acceptance/mod_proxy_html_spec.rb @@ -1,17 +1,7 @@ require 'spec_helper_acceptance' +require_relative './version.rb' describe 'apache::mod::proxy_html class' do - case fact('osfamily') - when 'Debian' - service_name = 'apache2' - when 'RedHat' - service_name = 'httpd' - when 'FreeBSD' - service_name = 'apache24' - when 'Gentoo' - service_name = 'apache2' - end - context "default proxy_html config" do if fact('osfamily') == 'RedHat' and fact('operatingsystemmajrelease') =~ /(5|6)/ it 'adds epel' do @@ -33,8 +23,12 @@ class { 'apache::mod::proxy_html': } apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end end diff --git a/apache/spec/acceptance/mod_security_spec.rb b/apache/spec/acceptance/mod_security_spec.rb index 4fcf0f551..d6f2987df 100644 --- a/apache/spec/acceptance/mod_security_spec.rb +++ b/apache/spec/acceptance/mod_security_spec.rb @@ -1,17 +1,7 @@ require 'spec_helper_acceptance' +require_relative './version.rb' describe 'apache::mod::security class', :unless => (fact('osfamily') == 'Debian' and (fact('lsbdistcodename') == 'squeeze' or fact('lsbdistcodename') == 'lucid' or fact('lsbdistcodename') == 'precise' or fact('lsbdistcodename') == 'wheezy')) do - case fact('osfamily') - when 'Debian' - mod_dir = '/etc/apache2/mods-available' - service_name = 'apache2' - package_name = 'apache2' - when 'RedHat' - mod_dir = '/etc/httpd/conf.d' - service_name = 'httpd' - package_name = 'httpd' - end - context "default mod_security config" do if fact('osfamily') == 'RedHat' and fact('operatingsystemmajrelease') =~ /(5|6)/ it 'adds epel' do @@ -54,16 +44,20 @@ class { 'apache::mod::security': } end end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end - describe package(package_name) do + describe package($package_name) do it { is_expected.to be_installed } end - describe file("#{mod_dir}/security.conf") do + describe file("#{$mod_dir}/security.conf") do it { is_expected.to contain "mod_security2.c" } end @@ -100,12 +94,16 @@ class { 'apache::mod::security': } apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end - describe file("#{mod_dir}/security.conf") do + describe file("#{$mod_dir}/security.conf") do it { is_expected.to contain "mod_security2.c" } end @@ -152,12 +150,16 @@ class { 'apache::mod::security': } apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end - describe file("#{mod_dir}/security.conf") do + describe file("#{$mod_dir}/security.conf") do it { is_expected.to contain "mod_security2.c" } end @@ -208,12 +210,16 @@ class { 'apache::mod::security': } apply_manifest(pp, :catch_failures => true) end - describe service(service_name) do - it { is_expected.to be_enabled } + describe service($service_name) do + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end - describe file("#{mod_dir}/security.conf") do + describe file("#{$mod_dir}/security.conf") do it { is_expected.to contain "mod_security2.c" } end diff --git a/apache/spec/acceptance/mod_suphp_spec.rb b/apache/spec/acceptance/mod_suphp_spec.rb index da2c6042e..fb23b504d 100644 --- a/apache/spec/acceptance/mod_suphp_spec.rb +++ b/apache/spec/acceptance/mod_suphp_spec.rb @@ -27,7 +27,11 @@ class { 'apache::mod::suphp': } end describe service('apache2') do - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end diff --git a/apache/spec/acceptance/nodesets/centos-59-x64.yml b/apache/spec/acceptance/nodesets/centos-59-x64.yml new file mode 100644 index 000000000..2ad90b86a --- /dev/null +++ b/apache/spec/acceptance/nodesets/centos-59-x64.yml @@ -0,0 +1,10 @@ +HOSTS: + centos-59-x64: + roles: + - master + platform: el-5-x86_64 + box : centos-59-x64-vbox4210-nocm + box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-59-x64-vbox4210-nocm.box + hypervisor : vagrant +CONFIG: + type: git diff --git a/apache/spec/acceptance/nodesets/centos-64-x64-pe.yml b/apache/spec/acceptance/nodesets/centos-64-x64-pe.yml new file mode 100644 index 000000000..7d9242f1b --- /dev/null +++ b/apache/spec/acceptance/nodesets/centos-64-x64-pe.yml @@ -0,0 +1,12 @@ +HOSTS: + centos-64-x64: + roles: + - master + - database + - dashboard + platform: el-6-x86_64 + box : centos-64-x64-vbox4210-nocm + box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210-nocm.box + hypervisor : vagrant +CONFIG: + type: pe diff --git a/apache/spec/acceptance/nodesets/centos-65-x64.yml b/apache/spec/acceptance/nodesets/centos-65-x64.yml new file mode 100644 index 000000000..4e2cb809e --- /dev/null +++ b/apache/spec/acceptance/nodesets/centos-65-x64.yml @@ -0,0 +1,10 @@ +HOSTS: + centos-65-x64: + roles: + - master + platform: el-6-x86_64 + box : centos-65-x64-vbox436-nocm + box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-nocm.box + hypervisor : vagrant +CONFIG: + type: foss diff --git a/apache/spec/acceptance/nodesets/debian-82-x64.yml b/apache/spec/acceptance/nodesets/debian-82-x64.yml new file mode 100644 index 000000000..800c49aaa --- /dev/null +++ b/apache/spec/acceptance/nodesets/debian-82-x64.yml @@ -0,0 +1,10 @@ +HOSTS: + debian-82: + roles: + - master + platform: debian-8-amd64 + box: puppetlabs/debian-8.2-64-nocm + hypervisor: vagrant +CONFIG: + log_level: debug + type: git diff --git a/apache/spec/acceptance/prefork_worker_spec.rb b/apache/spec/acceptance/prefork_worker_spec.rb index 234b6acef..668716144 100644 --- a/apache/spec/acceptance/prefork_worker_spec.rb +++ b/apache/spec/acceptance/prefork_worker_spec.rb @@ -1,15 +1,5 @@ require 'spec_helper_acceptance' - -case fact('osfamily') -when 'RedHat' - servicename = 'httpd' -when 'Debian' - servicename = 'apache2' -when 'FreeBSD' - servicename = 'apache24' -when 'Gentoo' - servicename = 'apache2' -end +require_relative './version.rb' case fact('osfamily') when 'FreeBSD' @@ -29,9 +19,13 @@ class { 'apache': end end - describe service(servicename) do + describe service($service_name) do it { is_expected.to be_running } - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end end end end @@ -39,43 +33,48 @@ class { 'apache': describe 'apache::mod::worker class' do describe 'running puppet code' do # Using puppet_apply as a helper - it 'should work with no errors' do - pp = <<-EOS + let(:pp) do + <<-EOS class { 'apache': mpm_module => 'worker', } EOS - - # Run it twice and test for idempotency - apply_manifest(pp, :catch_failures => true) - expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero end + + # Run it twice and test for idempotency + it_behaves_like "a idempotent resource" end - describe service(servicename) do + describe service($service_name) do it { is_expected.to be_running } - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end end end describe 'apache::mod::prefork class' do describe 'running puppet code' do # Using puppet_apply as a helper - it 'should work with no errors' do - pp = <<-EOS + let(:pp) do + <<-EOS class { 'apache': mpm_module => 'prefork', } EOS - - # Run it twice and test for idempotency - apply_manifest(pp, :catch_failures => true) - expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero end + # Run it twice and test for idempotency + it_behaves_like "a idempotent resource" end - describe service(servicename) do + describe service($service_name) do it { is_expected.to be_running } - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end end end diff --git a/apache/spec/acceptance/service_spec.rb b/apache/spec/acceptance/service_spec.rb index c3124c846..c62a34973 100644 --- a/apache/spec/acceptance/service_spec.rb +++ b/apache/spec/acceptance/service_spec.rb @@ -2,18 +2,17 @@ describe 'apache::service class' do describe 'adding dependencies in between the base class and service class' do - it 'should work with no errors' do - pp = <<-EOS - class { 'apache': } - file { '/tmp/test': - require => Class['apache'], - notify => Class['apache::service'], - } + let(:pp) do + <<-EOS + class { 'apache': } + file { '/tmp/test': + require => Class['apache'], + notify => Class['apache::service'], + } EOS - - # Run it twice and test for idempotency - apply_manifest(pp, :catch_failures => true) - expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero end + + # Run it twice and test for idempotency + it_behaves_like "a idempotent resource" end end diff --git a/apache/spec/acceptance/version.rb b/apache/spec/acceptance/version.rb index 117e23d9f..88cf509b7 100644 --- a/apache/spec/acceptance/version.rb +++ b/apache/spec/acceptance/version.rb @@ -5,7 +5,6 @@ case _osfamily when 'RedHat' $confd_dir = '/etc/httpd/conf.d' - $mod_dir = '/etc/httpd/conf.d' $conf_file = '/etc/httpd/conf/httpd.conf' $ports_file = '/etc/httpd/conf/ports.conf' $vhost_dir = '/etc/httpd/conf.d' @@ -19,8 +18,10 @@ if (_operatingsystem == 'Fedora' and _operatingsystemrelease >= 18) or (_operatingsystem != 'Fedora' and _operatingsystemrelease >= 7) $apache_version = '2.4' + $mod_dir = '/etc/httpd/conf.modules.d' else $apache_version = '2.2' + $mod_dir = '/etc/httpd/conf.d' end when 'Debian' $confd_dir = '/etc/apache2/conf.d' diff --git a/apache/spec/acceptance/vhost_spec.rb b/apache/spec/acceptance/vhost_spec.rb index 5d0294130..90b42e0e6 100644 --- a/apache/spec/acceptance/vhost_spec.rb +++ b/apache/spec/acceptance/vhost_spec.rb @@ -173,7 +173,11 @@ class { 'apache': } end describe service($service_name) do - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -213,7 +217,11 @@ class { 'apache': end describe service($service_name) do - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -265,7 +273,11 @@ class { 'apache': end describe service($service_name) do - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -322,7 +334,11 @@ class { 'apache': } end describe service($service_name) do - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -380,7 +396,11 @@ class { 'apache': } end describe service($service_name) do - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -414,7 +434,11 @@ class { 'apache': } end describe service($service_name) do - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -488,7 +512,11 @@ class { 'apache': } end describe service($service_name) do - it { should be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { should be_running } end @@ -523,7 +551,11 @@ class { 'apache': } end describe service($service_name) do - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -564,7 +596,11 @@ class { 'apache': } end describe service($service_name) do - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -610,7 +646,11 @@ class { 'apache': default_vhost => false, } end describe service($service_name) do - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -651,7 +691,11 @@ class { 'apache': default_vhost => false, } end describe service($service_name) do - it { is_expected.to be_enabled } + if (fact('operatingsystem') == 'Debian' && fact('operatingsystemmajrelease') == '8') + pending 'Should be enabled - Bug 760616 on Debian 8' + else + it { should be_enabled } + end it { is_expected.to be_running } end @@ -750,6 +794,8 @@ class { 'apache': } it { is_expected.not_to contain 'NameVirtualHost test.server' } elsif fact('operatingsystem') == 'Ubuntu' and fact('operatingsystemrelease') =~ /(14\.04|13\.10)/ it { is_expected.not_to contain 'NameVirtualHost test.server' } + elsif fact('operatingsystem') == 'Debian' and fact('operatingsystemmajrelease') == '8' + it { is_expected.not_to contain 'NameVirtualHost test.server' } else it { is_expected.to contain 'NameVirtualHost test.server' } end @@ -1013,7 +1059,7 @@ class { 'apache': service_ensure => stopped, } describe file("#{$vhost_dir}/25-test.server.conf") do it { is_expected.to be_file } - it { is_expected.to contain 'ProxyPass / test2/' } + it { is_expected.to contain 'ProxyPass / test2/' } end end @@ -1060,6 +1106,33 @@ class { 'apache': service_ensure => stopped, } end end + # Passenger isn't even in EPEL on el-5 + if (fact('osfamily') == 'RedHat' and fact('operatingsystemmajrelease') != '5') + describe 'rack_base_uris' do + before :all do + pp = "if $::osfamily == 'RedHat' { include epel }" + apply_manifest(pp, :catch_failures => true) + end + + it 'applies cleanly' do + pp = <<-EOS + class { 'apache': } + host { 'test.server': ip => '127.0.0.1' } + apache::vhost { 'test.server': + docroot => '/tmp', + rack_base_uris => ['/test'], + } + EOS + apply_manifest(pp, :catch_failures => true) + end + + describe file("#{$vhost_dir}/25-test.server.conf") do + it { is_expected.to be_file } + it { is_expected.to contain 'RackBaseURI /test' } + end + end + end + describe 'no_proxy_uris' do it 'applies cleanly' do pp = <<-EOS @@ -1076,8 +1149,8 @@ class { 'apache': service_ensure => stopped, } describe file("#{$vhost_dir}/25-test.server.conf") do it { is_expected.to be_file } - it { is_expected.to contain 'ProxyPass / http://test2/' } it { is_expected.to contain 'ProxyPass http://test2/test !' } + it { is_expected.to contain 'ProxyPass / http://test2/' } end end @@ -1325,18 +1398,57 @@ class { 'apache': } end end - # So what does this work on? - if default['platform'] !~ /^(debian-(6|7)|el-(5|6|7))/ + # Limit testing to Debian, since Centos does not have fastcgi package. + case fact('osfamily') + when 'Debian' describe 'fastcgi' do it 'applies cleanly' do pp = <<-EOS - if $::operatingsystem == 'Ubuntu' and versioncmp($::operatingsystemrelease, '10.04') >= 0 { + unless $::operatingsystem == 'Ubuntu' and versioncmp($::operatingsystemrelease, '12.04') >= 0 { + $_os = $::operatingsystem + + if $_os == 'Ubuntu' { + $_location = "http://archive.ubuntu.com/" + $_security_location = "http://archive.ubuntu.com/" + $_release = $::lsbdistcodename + $_release_security = "${_release}-security" + $_repos = "main universe multiverse" + } else { + $_location = "http://httpredir.debian.org/debian/" + $_security_location = "http://security.debian.org/" + $_release = $::lsbdistcodename + $_release_security = "${_release}/updates" + $_repos = "main contrib non-free" + } + include ::apt - apt::ppa { 'multiverse': - before => Class['Apache::Mod::Fastcgi'], + apt::source { "${_os}_${_release}": + location => $_location, + release => $_release, + repos => $_repos, + include_src => false, + } + + apt::source { "${_os}_${_release}-updates": + location => $_location, + release => "${_release}-updates", + repos => $_repos, + include_src => false, + } + + apt::source { "${_os}_${_release}-security": + location => $_security_location, + release => $_release_security, + repos => $_repos, + include_src => false, } } + EOS + + #apt-get update may not run clean here. Should be OK. + apply_manifest(pp, :catch_failures => false) + pp2 = <<-EOS class { 'apache': } class { 'apache::mod::fastcgi': } host { 'test.server': ip => '127.0.0.1' } @@ -1347,7 +1459,7 @@ class { 'apache::mod::fastcgi': } fastcgi_dir => '/tmp/fast', } EOS - apply_manifest(pp, :catch_failures => true) + apply_manifest(pp2, :catch_failures => true, :acceptable_exit_codes => [0, 2]) end describe file("#{$vhost_dir}/25-test.server.conf") do diff --git a/apache/spec/classes/apache_spec.rb b/apache/spec/classes/apache_spec.rb index a76676452..34d56ba4f 100644 --- a/apache/spec/classes/apache_spec.rb +++ b/apache/spec/classes/apache_spec.rb @@ -504,7 +504,7 @@ it { is_expected.to contain_file("/opt/rh/root/etc/httpd/conf/httpd.conf").with( 'ensure' => 'file', 'notify' => 'Class[Apache::Service]', - 'require' => ['Package[httpd]', 'File[/etc/httpd/conf/ports.conf]'], + 'require' => ['Package[httpd]', 'Concat[/etc/httpd/conf/ports.conf]'], ) } end @@ -835,6 +835,16 @@ ) } end + context 'with a custom file_mode parameter' do + let :params do { + :file_mode => '0640' + } + end + it { is_expected.to contain_concat("/etc/httpd/conf/ports.conf").with( + 'mode' => '0640', + ) + } + end context 'default vhost defaults' do it { is_expected.to contain_apache__vhost('default').with_ensure('present') } it { is_expected.to contain_apache__vhost('default-ssl').with_ensure('absent') } diff --git a/apache/spec/classes/mod/itk_spec.rb b/apache/spec/classes/mod/itk_spec.rb index cd2e6c67a..27369f144 100644 --- a/apache/spec/classes/mod/itk_spec.rb +++ b/apache/spec/classes/mod/itk_spec.rb @@ -55,6 +55,55 @@ it { is_expected.to contain_file("/etc/apache2/mods-enabled/itk.load").with_ensure('link') } end end + context "on a RedHat OS" do + let :facts do + { + :osfamily => 'RedHat', + :operatingsystemrelease => '6', + :concat_basedir => '/dne', + :operatingsystem => 'RedHat', + :id => 'root', + :kernel => 'Linux', + :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', + :is_pe => false, + } + end + it { is_expected.to contain_class("apache::params") } + it { is_expected.not_to contain_apache__mod('itk') } + it { is_expected.to contain_file("/etc/httpd/conf.d/itk.conf").with_ensure('file') } + it { is_expected.to contain_package("httpd-itk") } + + context "with Apache version < 2.4" do + let :params do + { + :apache_version => '2.2', + } + end + + it { is_expected.to contain_file_line("/etc/sysconfig/httpd itk enable").with({ + 'require' => 'Package[httpd]', + }) + } + end + + context "with Apache version >= 2.4" do + let :pre_condition do + 'class { "apache": mpm_module => prefork, }' + end + + let :params do + { + :apache_version => '2.4', + } + end + + it { is_expected.to contain_file("/etc/httpd/conf.d/itk.load").with({ + 'ensure' => 'file', + 'content' => "LoadModule mpm_itk_module modules/mod_mpm_itk.so\n" + }) + } + end + end context "on a FreeBSD OS" do let :pre_condition do 'class { "apache": mpm_module => false, }' diff --git a/apache/spec/classes/mod/ldap_spec.rb b/apache/spec/classes/mod/ldap_spec.rb index 2b82d8d1b..f51cafd4f 100644 --- a/apache/spec/classes/mod/ldap_spec.rb +++ b/apache/spec/classes/mod/ldap_spec.rb @@ -32,12 +32,22 @@ it { is_expected.to contain_file('ldap.conf').with_content(/^LDAPTrustedGlobalCert CA_BASE64 ca\.pem$/) } end - context 'ldap_trusted_global_cert_file and ldap_trusted_global_cert_type params' do + context 'set multiple ldap params' do let(:params) {{ :ldap_trusted_global_cert_file => 'ca.pem', - :ldap_trusted_global_cert_type => 'CA_DER' + :ldap_trusted_global_cert_type => 'CA_DER', + :ldap_shared_cache_size => '500000', + :ldap_cache_entries => '1024', + :ldap_cache_ttl => '600', + :ldap_opcache_entries => '1024', + :ldap_opcache_ttl => '600' }} it { is_expected.to contain_file('ldap.conf').with_content(/^LDAPTrustedGlobalCert CA_DER ca\.pem$/) } + it { is_expected.to contain_file('ldap.conf').with_content(/^LDAPSharedCacheSize 500000$/) } + it { is_expected.to contain_file('ldap.conf').with_content(/^LDAPCacheEntries 1024$/) } + it { is_expected.to contain_file('ldap.conf').with_content(/^LDAPCacheTTL 600$/) } + it { is_expected.to contain_file('ldap.conf').with_content(/^LDAPOpCacheEntries 1024$/) } + it { is_expected.to contain_file('ldap.conf').with_content(/^LDAPOpCacheTTL 600$/) } end end #Debian diff --git a/apache/spec/classes/mod/passenger_spec.rb b/apache/spec/classes/mod/passenger_spec.rb index 34dcc5c6f..1cae95c45 100644 --- a/apache/spec/classes/mod/passenger_spec.rb +++ b/apache/spec/classes/mod/passenger_spec.rb @@ -57,6 +57,13 @@ end it { is_expected.to contain_file('passenger.conf').with_content(/^ PassengerPoolIdleTime 1200$/) } end + describe "with passenger_max_request_queue_size => 100" do + let :params do + { :passenger_max_request_queue_size => 100 } + end + it { is_expected.to contain_file('passenger.conf').with_content(/^ PassengerMaxRequestQueueSize 100$/) } + end + describe "with passenger_max_requests => 20" do let :params do { :passenger_max_requests => 20 } diff --git a/apache/spec/classes/mod/php_spec.rb b/apache/spec/classes/mod/php_spec.rb index 6814b8a8f..3aaa3d329 100644 --- a/apache/spec/classes/mod/php_spec.rb +++ b/apache/spec/classes/mod/php_spec.rb @@ -117,9 +117,13 @@ let :pre_condition do 'class { "apache": mpm_module => itk, }' end - it 'should raise an error' do - expect { expect(subject).to contain_class("apache::mod::itk") }.to raise_error Puppet::Error, /Unsupported osfamily RedHat/ - end + it { is_expected.to contain_class("apache::params") } + it { is_expected.to contain_class("apache::mod::itk") } + it { is_expected.to contain_apache__mod('php5') } + it { is_expected.to contain_package("php") } + it { is_expected.to contain_file("php5.load").with( + :content => "LoadModule php5_module modules/libphp5.so\n" + ) } end end describe "on a FreeBSD OS" do diff --git a/apache/spec/classes/mod/security_spec.rb b/apache/spec/classes/mod/security_spec.rb index 93f751ee4..ba0bb2f71 100644 --- a/apache/spec/classes/mod/security_spec.rb +++ b/apache/spec/classes/mod/security_spec.rb @@ -28,7 +28,7 @@ ) } it { should contain_package('mod_security_crs') } it { should contain_file('security.conf').with( - :path => '/etc/httpd/conf.d/security.conf' + :path => '/etc/httpd/conf.modules.d/security.conf' ) } it { should contain_file('/etc/httpd/modsecurity.d').with( :ensure => 'directory', diff --git a/apache/spec/defines/mod_spec.rb b/apache/spec/defines/mod_spec.rb index e4e984529..1697190a3 100644 --- a/apache/spec/defines/mod_spec.rb +++ b/apache/spec/defines/mod_spec.rb @@ -34,6 +34,20 @@ end end + describe "with file_mode set" do + let :pre_condition do + "class {'::apache': file_mode => '0640'}" + end + let :title do + 'spec_m' + end + it "should manage the module load file" do + is_expected.to contain_file('spec_m.load').with({ + :mode => '0640', + } ) + end + end + describe "with shibboleth module and package param passed" do # name/title for the apache::mod define let :title do diff --git a/apache/spec/defines/vhost_spec.rb b/apache/spec/defines/vhost_spec.rb index 9dd563a9c..9a3027604 100644 --- a/apache/spec/defines/vhost_spec.rb +++ b/apache/spec/defines/vhost_spec.rb @@ -154,6 +154,7 @@ 'ssl_verify_depth' => '3', 'ssl_options' => '+ExportCertData', 'ssl_openssl_conf_cmd' => 'DHParameters "foo.pem"', + 'ssl_proxy_verify' => 'require', 'ssl_proxy_check_peer_cn' => 'on', 'ssl_proxy_check_peer_name' => 'on', 'ssl_proxyengine' => true, @@ -187,6 +188,10 @@ 'provider' => 'files', 'require' => 'all granted', }, + { + 'path' => '*', + 'provider' => 'proxy', + }, { 'path' => '/var/www/files/indexed_directory', 'directoryindex' => 'disabled', 'options' => ['Indexes','FollowSymLinks','MultiViews'], @@ -219,14 +224,25 @@ 'path' => '/a', 'url' => 'http://backend-a/', 'keywords' => ['noquery', 'interpolate'], - 'reverse_cookies' => [{ - 'path' => '/a', - 'url' => 'http://backend-a/', - }], + 'reverse_cookies' => [ + { + 'path' => '/a', + 'url' => 'http://backend-a/', + }, + { + 'domain' => 'foo', + 'url' => 'http://foo', + } + ], 'params' => { 'retry' => '0', 'timeout' => '5' }, + 'options' => { + 'Require' =>'valid-user', + 'AuthType' =>'Kerberos', + 'AuthName' =>'"Kerberos Login"' + }, 'setenv' => ['proxy-nokeepalive 1','force-proxy-request-1.0 1'], } ], @@ -390,6 +406,8 @@ it { is_expected.to contain_concat__fragment('rspec.example.com-itk') } it { is_expected.to contain_concat__fragment('rspec.example.com-fallbackresource') } it { is_expected.to contain_concat__fragment('rspec.example.com-directories') } + it { is_expected.to contain_concat__fragment('rspec.example.com-directories').with( + :content => /^\s+$/ ) } it { is_expected.to contain_concat__fragment('rspec.example.com-directories').with( :content => /^\s+Require valid-user$/ ) } it { is_expected.to contain_concat__fragment('rspec.example.com-directories').with( @@ -425,6 +443,15 @@ /noquery interpolate/) } it { is_expected.to contain_concat__fragment('rspec.example.com-proxy').with_content( /ProxyPassReverseCookiePath\s+\/a\s+http:\/\//) } + it { is_expected.to contain_concat__fragment('rspec.example.com-proxy').with_content( + /ProxyPassReverseCookieDomain\s+foo\s+http:\/\/foo/) } + it { is_expected.to contain_concat__fragment('rspec.example.com-proxy').with_content( + /Require valid-user/) } + it { is_expected.to contain_concat__fragment('rspec.example.com-proxy').with_content( + /AuthType Kerberos/) } + it { is_expected.to contain_concat__fragment('rspec.example.com-proxy').with_content( + /AuthName "Kerberos Login"/) } + it { is_expected.to contain_concat__fragment('rspec.example.com-rack') } it { is_expected.to contain_concat__fragment('rspec.example.com-redirect') } it { is_expected.to contain_concat__fragment('rspec.example.com-rewrite') } @@ -540,6 +567,39 @@ it { is_expected.to_not contain_concat__fragment('NameVirtualHost [::1]:80') } end + context 'vhost with wildcard ip address' do + let :params do + { + 'port' => '80', + 'ip' => '*', + 'ip_based' => true, + 'servername' => 'example.com', + 'docroot' => '/var/www/html', + 'add_listen' => true, + 'ensure' => 'present' + } + end + let :facts do + { + :osfamily => 'RedHat', + :operatingsystemrelease => '7', + :concat_basedir => '/dne', + :operatingsystem => 'RedHat', + :id => 'root', + :kernel => 'Linux', + :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', + :kernelversion => '3.6.2', + :is_pe => false, + } + end + + it { is_expected.to compile } + it { is_expected.to contain_concat__fragment('rspec.example.com-apache-header').with( + :content => /[.\/m]*[.\/m]*$/ ) } + it { is_expected.to contain_concat__fragment('Listen *:80') } + it { is_expected.to_not contain_concat__fragment('NameVirtualHost *:80') } + end + context 'set only aliases' do let :params do { @@ -776,6 +836,18 @@ end end # access logs describe 'validation' do + let :default_facts do + { + :osfamily => 'RedHat', + :operatingsystemrelease => '6', + :concat_basedir => '/dne', + :operatingsystem => 'RedHat', + :id => 'root', + :kernel => 'Linux', + :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', + :is_pe => false, + } + end context 'bad ensure' do let :params do { @@ -876,6 +948,16 @@ let :facts do default_facts end it { expect { is_expected.to compile }.to raise_error } end + context 'empty rewrites' do + let :params do + { + 'docroot' => '/rspec/docroot', + 'rewrites' => [], + } + end + let :facts do default_facts end + it { is_expected.to compile } + end context 'bad suexec_user_group' do let :params do { diff --git a/apache/spec/spec_helper_acceptance.rb b/apache/spec/spec_helper_acceptance.rb index 6bfdefd71..307f7b710 100644 --- a/apache/spec/spec_helper_acceptance.rb +++ b/apache/spec/spec_helper_acceptance.rb @@ -16,6 +16,12 @@ # Readable test descriptions c.formatter = :documentation + # detect the situation where PUP-5016 is triggered and skip the idempotency tests in that case + # also note how fact('puppetversion') is not available because of PUP-4359 + if fact('osfamily') == 'Debian' && fact('operatingsystemmajrelease') == '8' && shell('puppet --version').stdout =~ /^4\.2/ + c.filter_run_excluding :skip_pup_5016 => true + end + # Configure all nodes in nodeset c.before :suite do # net-tools required for netstat utility being used by be_listening @@ -43,6 +49,12 @@ if fact('osfamily') == 'RedHat' on host, puppet('module','install','stahnma/epel') on host, puppet('module','install','puppetlabs/inifile') + #we need epel installed, so we can get plugins, wsgi, mime ... + pp = <<-EOS + class { 'epel': } + EOS + + apply_manifest_on(host, pp, :catch_failures => true) end # Required for manifest to make mod_pagespeed repository available @@ -56,3 +68,13 @@ end end end + +shared_examples "a idempotent resource" do + it 'should apply with no errors' do + apply_manifest(pp, :catch_failures => true) + end + + it 'should apply a second time without changes', :skip_pup_5016 do + apply_manifest(pp, :catch_changes => true) + end +end diff --git a/apache/spec/unit/apache_version_spec.rb b/apache/spec/unit/apache_version_spec.rb new file mode 100644 index 000000000..30f6ef991 --- /dev/null +++ b/apache/spec/unit/apache_version_spec.rb @@ -0,0 +1,20 @@ +require 'spec_helper' + +describe Facter::Util::Fact do + before do + Facter.clear + end + + describe 'apache_version' do + context 'with value' do + before :each do + Facter::Util::Resolution.stubs(:which).with('apachectl').returns(true) + Facter::Util::Resolution.stubs(:exec).with('apachectl -v 2>&1').returns('Server version: Apache/2.4.16 (Unix) + Server built: Jul 31 2015 15:53:26') + end + it do + expect(Facter.fact(:apache_version).value).to eq('2.4.16') + end + end + end +end diff --git a/apache/spec/unit/puppet/parser/functions/enclose_ipv6_spec.rb b/apache/spec/unit/puppet/parser/functions/enclose_ipv6_spec.rb index 508f62aea..b162127d0 100644 --- a/apache/spec/unit/puppet/parser/functions/enclose_ipv6_spec.rb +++ b/apache/spec/unit/puppet/parser/functions/enclose_ipv6_spec.rb @@ -28,6 +28,10 @@ expect { scope.function_enclose_ipv6(['127.0.0.1']) }.to_not raise_error end + it "should not raise a ParseError when given * as ip string" do + expect { scope.function_enclose_ipv6(['*']) }.to_not raise_error + end + it "should not raise a ParseError when given an array of ip strings" do expect { scope.function_enclose_ipv6([['127.0.0.1','fe80::1']]) }.to_not raise_error end @@ -49,8 +53,8 @@ end it "should embrace ipv6 adresses within an array of ip addresses" do - result = scope.function_enclose_ipv6([['127.0.0.1','fe80::1','[fe80::1]']]) - expect(result).to(eq(['127.0.0.1','[fe80::1]','[fe80::1]'])) + result = scope.function_enclose_ipv6([['127.0.0.1','fe80::1','[fe80::2]']]) + expect(result).to(eq(['127.0.0.1','[fe80::1]','[fe80::2]'])) end it "should embrace a single ipv6 adresse" do diff --git a/apache/templates/httpd.conf.erb b/apache/templates/httpd.conf.erb index cc6998b9b..9c854cfc3 100644 --- a/apache/templates/httpd.conf.erb +++ b/apache/templates/httpd.conf.erb @@ -77,6 +77,9 @@ LogFormat "%{Referer}i -> %U" referer <% unless @log_formats.has_key?('agent') -%> LogFormat "%{User-agent}i" agent <% end -%> +<% unless @log_formats.has_key?('forwarded') -%> +LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"" forwarded +<% end -%> <% if @log_formats and !@log_formats.empty? -%> <%- @log_formats.sort.each do |nickname,format| -%> LogFormat "<%= format -%>" <%= nickname %> diff --git a/apache/templates/mod/ldap.conf.erb b/apache/templates/mod/ldap.conf.erb index fbb4b9213..424fbe8ee 100644 --- a/apache/templates/mod/ldap.conf.erb +++ b/apache/templates/mod/ldap.conf.erb @@ -12,3 +12,18 @@ <% if @ldap_trusted_global_cert_file -%> LDAPTrustedGlobalCert <%= @ldap_trusted_global_cert_type %> <%= @ldap_trusted_global_cert_file %> <% end -%> +<%- if @ldap_shared_cache_size -%> +LDAPSharedCacheSize <%= @ldap_shared_cache_size %> +<%- end -%> +<%- if @ldap_cache_entries -%> +LDAPCacheEntries <%= @ldap_cache_entries %> +<%- end -%> +<%- if @ldap_cache_ttl -%> +LDAPCacheTTL <%= @ldap_cache_ttl %> +<%- end -%> +<%- if @ldap_opcache_entries -%> +LDAPOpCacheEntries <%= @ldap_opcache_entries %> +<%- end -%> +<%- if @ldap_opcache_ttl -%> +LDAPOpCacheTTL <%= @ldap_opcache_ttl %> +<%- end -%> diff --git a/apache/templates/mod/passenger.conf.erb b/apache/templates/mod/passenger.conf.erb index 9bd13c22a..8a3e9d4f3 100644 --- a/apache/templates/mod/passenger.conf.erb +++ b/apache/templates/mod/passenger.conf.erb @@ -22,6 +22,9 @@ <%- if @passenger_pool_idle_time -%> PassengerPoolIdleTime <%= @passenger_pool_idle_time %> <%- end -%> + <%- if @passenger_max_request_queue_size -%> + PassengerMaxRequestQueueSize <%= @passenger_max_request_queue_size %> + <%- end -%> <%- if @passenger_max_requests -%> PassengerMaxRequests <%= @passenger_max_requests %> <%- end -%> diff --git a/apache/templates/mod/security.conf.erb b/apache/templates/mod/security.conf.erb index 7b2da7613..a71f5887d 100644 --- a/apache/templates/mod/security.conf.erb +++ b/apache/templates/mod/security.conf.erb @@ -37,8 +37,8 @@ SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" \ "id:'200003',phase:2,t:none,log,deny,status:44,msg:'Multipart parser detected a possible unmatched boundary.'" - SecPcreMatchLimit 1000 - SecPcreMatchLimitRecursion 1000 + SecPcreMatchLimit <%= @secpcrematchlimit %> + SecPcreMatchLimitRecursion <%= @secpcrematchlimitrecursion %> SecRule TX:/^MSC_/ "!@streq 0" \ "id:'200004',phase:2,t:none,deny,msg:'ModSecurity internal error flagged: %{MATCHED_VAR_NAME}'" diff --git a/apache/templates/mod/worker.conf.erb b/apache/templates/mod/worker.conf.erb index ad2bc4461..8ad6451c7 100644 --- a/apache/templates/mod/worker.conf.erb +++ b/apache/templates/mod/worker.conf.erb @@ -1,11 +1,11 @@ ServerLimit <%= @serverlimit %> StartServers <%= @startservers %> + ThreadLimit <%= @threadlimit %> MaxClients <%= @maxclients %> MinSpareThreads <%= @minsparethreads %> MaxSpareThreads <%= @maxsparethreads %> ThreadsPerChild <%= @threadsperchild %> MaxRequestsPerChild <%= @maxrequestsperchild %> - ThreadLimit <%= @threadlimit %> ListenBacklog <%= @listenbacklog %> diff --git a/apache/templates/vhost/_access_log.erb b/apache/templates/vhost/_access_log.erb index d1ec426a4..894daa7ce 100644 --- a/apache/templates/vhost/_access_log.erb +++ b/apache/templates/vhost/_access_log.erb @@ -10,7 +10,7 @@ <% destination = "#{@logroot}/#{log['file']}" -%> <% end -%> <% elsif log['syslog'] -%> -<% destination = "syslog" -%> +<% destination = log['syslog'] -%> <% elsif log['pipe'] -%> <% destination = log['pipe'] -%> <% else -%> diff --git a/apache/templates/vhost/_custom_fragment.erb b/apache/templates/vhost/_custom_fragment.erb index 973964655..35c264adb 100644 --- a/apache/templates/vhost/_custom_fragment.erb +++ b/apache/templates/vhost/_custom_fragment.erb @@ -1,5 +1,5 @@ <% if @custom_fragment -%> ## Custom fragment -<%= @custom_fragment %> + <%= @custom_fragment %> <% end -%> diff --git a/apache/templates/vhost/_directories.erb b/apache/templates/vhost/_directories.erb index c95fda5a1..49a9bd901 100644 --- a/apache/templates/vhost/_directories.erb +++ b/apache/templates/vhost/_directories.erb @@ -17,7 +17,7 @@ <%- end -%> <%- end -%> <%- if directory['path'] and directory['path'] != '' -%> - <%- if directory['provider'] and directory['provider'].match('(directory|location|files)') -%> + <%- if directory['provider'] and directory['provider'].match('(directory|location|files|proxy)') -%> <%- if /^(.*)match$/ =~ directory['provider'] -%> <%- provider = $1.capitalize + 'Match' -%> <%- else -%> diff --git a/apache/templates/vhost/_proxy.erb b/apache/templates/vhost/_proxy.erb index 3f94af911..4e36361ca 100644 --- a/apache/templates/vhost/_proxy.erb +++ b/apache/templates/vhost/_proxy.erb @@ -5,6 +5,8 @@ <%- end -%> <% if @proxy_preserve_host -%> ProxyPreserveHost On +<% else -%> + ProxyPreserveHost Off <%- end -%> <% if @proxy_error_override -%> ProxyErrorOverride On @@ -12,69 +14,71 @@ <%- [@proxy_pass].flatten.compact.each do |proxy| -%> ProxyPass <%= proxy['path'] %> <%= proxy['url'] -%> <%- if proxy['params'] -%> - <%- proxy['params'].each_pair do |key, value| -%> <%= key %>=<%= value -%> + <%- proxy['params'].keys.sort.each do |key| -%> <%= key %>=<%= proxy['params'][key] -%> <%- end -%> <%- end -%> <%- if proxy['keywords'] %> <%= proxy['keywords'].join(' ') -%> <%- end %> - > <%- if not proxy['reverse_cookies'].nil? -%> <%- Array(proxy['reverse_cookies']).each do |reverse_cookies| -%> - ProxyPassReverseCookiePath <%= reverse_cookies['path'] %> <%= reverse_cookies['url'] %> + <%- if reverse_cookies['path'] -%> + ProxyPassReverseCookiePath <%= reverse_cookies['path'] %> <%= reverse_cookies['url'] %> + <%- end -%> + <%- if reverse_cookies['domain'] -%> + ProxyPassReverseCookieDomain <%= reverse_cookies['domain'] %> <%= reverse_cookies['url'] %> + <%- end -%> <%- end -%> <%- end -%> <%- if proxy['reverse_urls'].nil? -%> - ProxyPassReverse <%= proxy['url'] %> + ProxyPassReverse <%= proxy['path'] %> <%= proxy['url'] %> <%- else -%> <%- Array(proxy['reverse_urls']).each do |reverse_url| -%> - ProxyPassReverse <%= reverse_url %> + ProxyPassReverse <%= proxy['path'] %> <%= reverse_url %> <%- end -%> <%- end -%> <%- if proxy['setenv'] -%> <%- Array(proxy['setenv']).each do |setenv_var| -%> - SetEnv <%= setenv_var %> + SetEnv <%= setenv_var %> + <%- end -%> + <%- end -%> + <%- if proxy['options'] -%> + <%- proxy['options'].keys.sort.each do |key| -%> + <%= key %> <%= proxy['options'][key] %> <%- end -%> <%- end -%> - <% end -%> <% [@proxy_pass_match].flatten.compact.each do |proxy| %> ProxyPassMatch <%= proxy['path'] %> <%= proxy['url'] -%> <%- if proxy['params'] -%> - <%- proxy['params'].each_pair do |key, value| -%> <%= key %>=<%= value -%> + <%- proxy['params'].keys.sort.each do |key| -%> <%= key %>=<%= proxy['params'][key] -%> <%- end -%> <%- end -%> <%- if proxy['keywords'] %> <%= proxy['keywords'].join(' ') -%> <%- end %> - > <%- if proxy['reverse_urls'].nil? -%> - ProxyPassReverse <%= proxy['url'] %> + ProxyPassReverse <%= proxy['path'] %> <%= proxy['url'] %> <%- else -%> <%- Array(proxy['reverse_urls']).each do |reverse_url| -%> - ProxyPassReverse <%= reverse_url %> + ProxyPassReverse <%= proxy['path'] %> <%= reverse_url %> <%- end -%> <%- end -%> <%- if proxy['setenv'] -%> <%- Array(proxy['setenv']).each do |setenv_var| -%> - SetEnv <%= setenv_var %> + SetEnv <%= setenv_var %> <%- end -%> <%- end -%> - <% end -%> <% if @proxy_dest -%> <%- Array(@no_proxy_uris).each do |uri| -%> ProxyPass <%= uri %> ! <% end -%> - ProxyPass / <%= @proxy_dest %>/ - - ProxyPassReverse <%= @proxy_dest %>/ - + ProxyPass / <%= @proxy_dest %>/ + ProxyPassReverse / <%= @proxy_dest %>/ <% end -%> <% if @proxy_dest_match -%> <%- Array(@no_proxy_uris_match).each do |uri| -%> - ProxyPassMatch <%= uri %> ! + ProxyPassMatch <%= uri %> ! <% end -%> - ProxyPassMatch / <%= @proxy_dest_match %>/ - - ProxyPassReverse <%= @proxy_dest_reverse_match %>/ - + ProxyPassMatch / <%= @proxy_dest_match %>/ + ProxyPassReverse / <%= @proxy_dest_reverse_match %>/ <% end -%> diff --git a/apache/templates/vhost/_redirect.erb b/apache/templates/vhost/_redirect.erb index 69bbfd09d..209da646c 100644 --- a/apache/templates/vhost/_redirect.erb +++ b/apache/templates/vhost/_redirect.erb @@ -22,4 +22,14 @@ <% @redirectmatch_dest_a[i] ||= @redirectmatch_dest_a[0] -%> RedirectMatch <%= "#{@redirectmatch_status_a[i]} " %> <%= @redirectmatch_regexp_a[i] %> <%= @redirectmatch_dest_a[i] %> <%- end -%> +<%- elsif @redirectmatch_regexp and @redirectmatch_dest -%> +<% @redirectmatch_regexp_a = Array(@redirectmatch_regexp) -%> +<% @redirectmatch_dest_a = Array(@redirectmatch_dest) -%> + + ## RedirectMatch rules + <%- @redirectmatch_regexp_a.each_with_index do |status, i| -%> +<% @redirectmatch_regexp_a[i] ||= @redirectmatch_regexp_a[0] -%> +<% @redirectmatch_dest_a[i] ||= @redirectmatch_dest_a[0] -%> + RedirectMatch <%= @redirectmatch_regexp_a[i] %> <%= @redirectmatch_dest_a[i] %> + <%- end -%> <% end -%> diff --git a/apache/templates/vhost/_sslproxy.erb b/apache/templates/vhost/_sslproxy.erb index e58d52d0b..568d9d1d0 100644 --- a/apache/templates/vhost/_sslproxy.erb +++ b/apache/templates/vhost/_sslproxy.erb @@ -2,6 +2,9 @@ # SSL Proxy directives SSLProxyEngine On + <%- if @ssl_proxy_verify -%> + SSLProxyVerify <%= @ssl_proxy_verify %> + <%- end -%> <%- if @ssl_proxy_check_peer_cn -%> SSLProxyCheckPeerCN <%= @ssl_proxy_check_peer_cn %> <%- end -%> From 626bd9babb1e4698960cfffe46da1a01ccf1131a Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Fri, 5 Feb 2016 09:07:18 -0500 Subject: [PATCH 25/35] Weekly liberty sync of openstack modules. Update aodh to 0cf4beb29311515d51dfa1c5cc483311f0c219ef 0cf4beb29311515d51dfa1c5cc483311f0c219ef drop mock dependency Change-Id: Ib6a542be51486e3ef30e2b8929e840a6a14ce297 Update ceilometer to 4446631c8928c4eb7d29e90e3ecbba119a6f1ec4 4446631c8928c4eb7d29e90e3ecbba119a6f1ec4 drop mock dependency Change-Id: Id141ce76a6e810f5b16546bb7c2553b6a06da09c Update cinder to 4e26e2909b8d41dede6afb15321d455c78c445a1 4e26e2909b8d41dede6afb15321d455c78c445a1 drop mock dependency Change-Id: I1bdf3d7b93390fce22f1809bb171b4d7ae2e799c Update glance to 5160235bf1246295cfca22e2ee0d93f79bfe07b7 5160235bf1246295cfca22e2ee0d93f79bfe07b7 Use glance-swift conf for swift backend 457b4c1b6b24421fe6a013089cf115e126a5c40c drop mock dependency Change-Id: I48904c739d56fa0aa0ec559983f4f7531acde7d7 Update heat to 043a1ea5dcd60b7a626202a1b86b98c9c1f9a90f 043a1ea5dcd60b7a626202a1b86b98c9c1f9a90f drop mock dependency Change-Id: I2a5b17f46c4eafab7106c5c71e9295a452ee00e5 Update horizon to fc9f99b66cf36ad868c392691254e8d8855b578b fc9f99b66cf36ad868c392691254e8d8855b578b drop mock dependency Change-Id: I2d62b66b27ab3634fd3b221d5b8e6a8f7686238a Update keystone to 5525e033af2cd70d7efc3b61ebc99e20d52fc882 5525e033af2cd70d7efc3b61ebc99e20d52fc882 Merge "Follow-up on PyMySQL support for Red Hat platforms" into stable/liberty 4a0a3ed7d1fed8c84098e78616e1d0cd15bdad00 sanitize IPv6 in keystone provider 83709d18a9474f7326c1f01571be82092fa5f038 Follow-up on PyMySQL support for Red Hat platforms Change-Id: I9814f1b030d9544429e2aee9c6a1a604294965ac Update manila to 28b0b86e8eb19230f162a51d167ce30093bb9bc6 28b0b86e8eb19230f162a51d167ce30093bb9bc6 drop mock dependency Change-Id: I1c4803756b7887181747ff75d4f5db75e7d3c27e Update neutron to 3794e88c4e6cbf3a14125f19a1be9fa862718583 3794e88c4e6cbf3a14125f19a1be9fa862718583 Correctly parse csv even with extra output. e95d00d7b6d5f25b21d84928cee6cff5f2f6ea3e Neutron parses the wrong json revert to shell/csv. 998cb49b0c611a3b99dc7162bf481a291ad364a6 Add puppet files to support big switch agents ee3175f227f4c252bb4b31456dc6fb2d4e738953 drop mock dependency Change-Id: Ib9591caa8a76dac87094326ebdea5b6c5842b094 Update nova to bc3eda43a6fa59334414dd5c1916fad8b5f99a57 bc3eda43a6fa59334414dd5c1916fad8b5f99a57 drop mock dependency Change-Id: Ie471bb950babe8f55effe5fc0edd6984434ba2d5 Update tempest to b7aaccb9243a889a3ac8445b26168f720781ed4a b7aaccb9243a889a3ac8445b26168f720781ed4a Switch id setters to openstack client d52158ff5b1419bdd2c5ba3ee5580e8b84cdefe3 drop mock dependency Change-Id: I246db02a9d8cd292cfc823f2ea076f73c25e5058 Update tripleo to 78e62660da958a975a899c5866a89d8158020222 78e62660da958a975a899c5866a89d8158020222 Enable X-Forwarded-Proto header for keystone_public 5fa00e27a29c98c985b74011f9903b2947f7e088 Drop webmock dependency Change-Id: I73f44d678fd40926dba39eefc1e9c34d70164139 Update trove to dac27f13f2d376d09511811ea2c84f8b06cd969d dac27f13f2d376d09511811ea2c84f8b06cd969d drop mock dependency Change-Id: I367a6070d829ea4ff02e62923b0f2a55ea08721b Update openstacklib to d36b88673a893284d5898159cc39ab93bc18c00c d36b88673a893284d5898159cc39ab93bc18c00c drop mock dependency Change-Id: Id141b9b658ae9e4c71419bfda42d0cd30f4eecc7 Update openstack_extras to 9dbbbe3f40d62e391e1dd682975cd5fed8b6d64a 9dbbbe3f40d62e391e1dd682975cd5fed8b6d64a drop mock dependency Change-Id: I4c218b3b066835b0a9c49e3912960d28e06deb90 Update vswitch to f8aba9c28ffc49e2de34452aefb17fa85296ff5d f8aba9c28ffc49e2de34452aefb17fa85296ff5d Fix vs_port usage in Red Hat distros 8dc2c0938811fa4df43472a803fb7cc834839efb drop mock dependency Change-Id: I4d1cce6ce7cf498ce67e38b89455a21d83105db3 --- Puppetfile | 32 ++-- aodh/Gemfile | 1 - aodh/spec/classes/aodh_evaluator_spec.rb | 2 - aodh/spec/classes/aodh_listener_spec.rb | 2 - aodh/spec/classes/aodh_notifier_spec.rb | 2 - aodh/spec/spec_helper.rb | 1 - ceilometer/Gemfile | 1 - .../spec/classes/ceilometer_collector_spec.rb | 2 - ceilometer/spec/spec_helper.rb | 1 - cinder/Gemfile | 1 - cinder/spec/spec_helper.rb | 1 - glance/Gemfile | 1 - .../glance_swift_config/ini_setting.rb | 10 ++ glance/lib/puppet/type/glance_swift_config.rb | 57 +++++++ glance/manifests/backend/swift.pp | 5 +- .../spec/classes/glance_backend_swift_spec.rb | 18 +- glance/spec/spec_helper.rb | 1 - .../glance_swift_config/ini_setting_spec.rb | 60 +++++++ .../unit/type/glance_swift_config_spec.rb | 41 +++++ heat/Gemfile | 1 - heat/spec/spec_helper.rb | 1 - horizon/Gemfile | 1 - horizon/spec/spec_helper.rb | 1 - keystone/lib/puppet/provider/keystone.rb | 7 +- keystone/manifests/params.pp | 2 +- keystone/spec/classes/keystone_db_spec.rb | 31 ++-- keystone/spec/unit/provider/keystone_spec.rb | 8 + manila/Gemfile | 1 - manila/spec/spec_helper.rb | 1 - neutron/Gemfile | 1 - neutron/lib/puppet/provider/neutron.rb | 98 +++++------ neutron/lib/puppet/type/neutron_plugin_ml2.rb | 22 +++ neutron/manifests/agents/bigswitch.pp | 71 ++++++++ neutron/manifests/params.pp | 5 + .../plugins/ml2/bigswitch/restproxy.pp | 26 +++ .../spec/classes/neutron_agents_bigswitch.rb | 114 +++++++++++++ ...on_plugins_ml2_bigswitch_restproxy_spec.rb | 10 ++ neutron/spec/spec_helper.rb | 1 - .../provider/neutron_router/neutron_spec.rb | 18 +- neutron/spec/unit/provider/neutron_spec.rb | 160 ++++++------------ nova/Gemfile | 3 +- nova/spec/classes/nova_vnc_proxy_spec.rb | 2 - nova/spec/spec_helper.rb | 1 - openstack_extras/Gemfile | 1 - openstack_extras/spec/spec_helper.rb | 1 - openstacklib/Gemfile | 1 - openstacklib/spec/spec_helper.rb | 1 - tempest/Gemfile | 1 - tempest/lib/puppet/provider/tempest.rb | 44 +++++ .../{ruby.rb => openstack.rb} | 40 ++++- .../{ruby.rb => openstack.rb} | 40 ++++- tempest/manifests/init.pp | 3 + tempest/metadata.json | 3 +- tempest/spec/spec_helper.rb | 1 - .../spec/unit/puppet/provider/ruby_spec.rb | 4 +- tripleo/Gemfile | 1 - tripleo/manifests/loadbalancer.pp | 4 + tripleo/spec/spec_helper.rb | 1 - trove/Gemfile | 1 - trove/spec/spec_helper.rb | 1 - vswitch/Gemfile | 1 - .../lib/puppet/provider/vs_port/ovs_redhat.rb | 4 +- vswitch/spec/spec_helper.rb | 1 - 63 files changed, 704 insertions(+), 275 deletions(-) create mode 100644 glance/lib/puppet/provider/glance_swift_config/ini_setting.rb create mode 100644 glance/lib/puppet/type/glance_swift_config.rb create mode 100644 glance/spec/unit/provider/glance_swift_config/ini_setting_spec.rb create mode 100644 glance/spec/unit/type/glance_swift_config_spec.rb create mode 100644 neutron/manifests/agents/bigswitch.pp create mode 100644 neutron/spec/classes/neutron_agents_bigswitch.rb create mode 100644 tempest/lib/puppet/provider/tempest.rb rename tempest/lib/puppet/provider/tempest_glance_id_setter/{ruby.rb => openstack.rb} (53%) rename tempest/lib/puppet/provider/tempest_neutron_net_id_setter/{ruby.rb => openstack.rb} (53%) diff --git a/Puppetfile b/Puppetfile index 97f05290d..cd160fe26 100644 --- a/Puppetfile +++ b/Puppetfile @@ -1,5 +1,5 @@ mod 'aodh', - :commit => '5c5478104458d7e682df71036d6af2d571d21249', + :commit => '0cf4beb29311515d51dfa1c5cc483311f0c219ef', :git => 'https://github.com/openstack/puppet-aodh.git' mod 'apache', @@ -11,7 +11,7 @@ mod 'cassandra', :git => 'https://github.com/locp/cassandra.git' mod 'ceilometer', - :commit => 'f27e53e9903d847fb5c14685d8ff009319c7ea8f', + :commit => '4446631c8928c4eb7d29e90e3ecbba119a6f1ec4', :git => 'https://github.com/openstack/puppet-ceilometer.git' mod 'ceph', @@ -23,7 +23,7 @@ mod 'certmonger', :git => 'https://github.com/rcritten/puppet-certmonger.git' mod 'cinder', - :commit => '40d5f3e385a54e7626fc14cabccaec6eebcad66b', + :commit => '4e26e2909b8d41dede6afb15321d455c78c445a1', :git => 'https://github.com/openstack/puppet-cinder.git' mod 'common', @@ -67,7 +67,7 @@ mod 'git', :git => 'https://github.com/puppetlabs/puppetlabs-git.git' mod 'glance', - :commit => '66abdfea280a68edae92c291e194df542b5f64da', + :commit => '5160235bf1246295cfca22e2ee0d93f79bfe07b7', :git => 'https://github.com/openstack/puppet-glance.git' mod 'gluster', @@ -83,11 +83,11 @@ mod 'haproxy', :git => 'https://github.com/puppetlabs/puppetlabs-haproxy.git' mod 'heat', - :commit => '408dfb4d65992463d62fc7bafe8e1fc6b4f29a60', + :commit => '043a1ea5dcd60b7a626202a1b86b98c9c1f9a90f', :git => 'https://github.com/openstack/puppet-heat.git' mod 'horizon', - :commit => '0d1398f418ee89eb5cf8e2718a447420d4915e2f', + :commit => 'fc9f99b66cf36ad868c392691254e8d8855b578b', :git => 'https://github.com/openstack/puppet-horizon.git' mod 'inifile', @@ -111,7 +111,7 @@ mod 'keepalived', :git => 'https://github.com/Unyonsys/puppet-module-keepalived.git' mod 'keystone', - :commit => 'eed17a8460994382c14c2573e311c528fe3bd761', + :commit => '5525e033af2cd70d7efc3b61ebc99e20d52fc882', :git => 'https://github.com/openstack/puppet-keystone.git' mod 'kibana3', @@ -123,7 +123,7 @@ mod 'kmod', :git => 'https://github.com/camptocamp/puppet-kmod.git' mod 'manila', - :commit => '9d74004a388f65d8e54c8ab4dfb5832a3b024925', + :commit => '28b0b86e8eb19230f162a51d167ce30093bb9bc6', :git => 'https://github.com/openstack/puppet-manila.git' mod 'memcached', @@ -159,11 +159,11 @@ mod 'nagios', :git => 'https://github.com/gildub/puppet-nagios-openstack.git' mod 'neutron', - :commit => 'ade5a77e9be344234ade6267e3beca4cd2556b43', + :commit => '3794e88c4e6cbf3a14125f19a1be9fa862718583', :git => 'https://github.com/openstack/puppet-neutron.git' mod 'nova', - :commit => '2ec3e65e34a35e1099e07b79546e5b4de43c014a', + :commit => 'bc3eda43a6fa59334414dd5c1916fad8b5f99a57', :git => 'https://github.com/openstack/puppet-nova.git' mod 'nssdb', @@ -179,11 +179,11 @@ mod 'opendaylight', :git => 'https://github.com/dfarrell07/puppet-opendaylight.git' mod 'openstack_extras', - :commit => 'b689f85d4f8d81eb8d1d1fc26a9f48acf71c1c41', + :commit => '9dbbbe3f40d62e391e1dd682975cd5fed8b6d64a', :git => 'https://github.com/openstack/puppet-openstack_extras.git' mod 'openstacklib', - :commit => '247553cf382083fb85ca7987ee7d3cf208ec3e64', + :commit => 'd36b88673a893284d5898159cc39ab93bc18c00c', :git => 'https://github.com/openstack/puppet-openstacklib.git' mod 'pacemaker', @@ -247,7 +247,7 @@ mod 'sysctl', :git => 'https://github.com/puppetlabs/puppetlabs-sysctl.git' mod 'tempest', - :commit => 'a26f81393bd829ddbc8ca8e1e8de226538208f1e', + :commit => 'b7aaccb9243a889a3ac8445b26168f720781ed4a', :git => 'https://github.com/openstack/puppet-tempest.git' mod 'timezone', @@ -259,11 +259,11 @@ mod 'tomcat', :git => 'https://github.com/puppetlabs/puppetlabs-tomcat.git' mod 'tripleo', - :commit => 'e40d5282833ddfa2c700044440b7c96101736453', + :commit => '78e62660da958a975a899c5866a89d8158020222', :git => 'https://github.com/openstack/puppet-tripleo.git' mod 'trove', - :commit => '78eaf20cb21e6ff29569701847b6d8d448e9dfe5', + :commit => 'dac27f13f2d376d09511811ea2c84f8b06cd969d', :git => 'https://github.com/openstack/puppet-trove' mod 'tuskar', @@ -283,7 +283,7 @@ mod 'vlan', :git => 'https://github.com/derekhiggins/puppet-vlan.git' mod 'vswitch', - :commit => '906abc6dd77218e8734a9e7c6101beb951d4d627', + :commit => 'f8aba9c28ffc49e2de34452aefb17fa85296ff5d', :git => 'https://github.com/openstack/puppet-vswitch.git' mod 'xinetd', diff --git a/aodh/Gemfile b/aodh/Gemfile index fc2214398..984392186 100644 --- a/aodh/Gemfile +++ b/aodh/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/aodh/spec/classes/aodh_evaluator_spec.rb b/aodh/spec/classes/aodh_evaluator_spec.rb index 7ada65f62..d868dc5dc 100644 --- a/aodh/spec/classes/aodh_evaluator_spec.rb +++ b/aodh/spec/classes/aodh_evaluator_spec.rb @@ -1,6 +1,4 @@ require 'spec_helper' -# LP1492636 - Cohabitation of compile matcher and webmock -WebMock.disable_net_connect!(:allow => "169.254.169.254") describe 'aodh::evaluator' do diff --git a/aodh/spec/classes/aodh_listener_spec.rb b/aodh/spec/classes/aodh_listener_spec.rb index 180b5f342..dd722a468 100644 --- a/aodh/spec/classes/aodh_listener_spec.rb +++ b/aodh/spec/classes/aodh_listener_spec.rb @@ -1,6 +1,4 @@ require 'spec_helper' -# LP1492636 - Cohabitation of compile matcher and webmock -WebMock.disable_net_connect!(:allow => "169.254.169.254") describe 'aodh::listener' do diff --git a/aodh/spec/classes/aodh_notifier_spec.rb b/aodh/spec/classes/aodh_notifier_spec.rb index 94d7b9cce..f63115c69 100644 --- a/aodh/spec/classes/aodh_notifier_spec.rb +++ b/aodh/spec/classes/aodh_notifier_spec.rb @@ -1,6 +1,4 @@ require 'spec_helper' -# LP1492636 - Cohabitation of compile matcher and webmock -WebMock.disable_net_connect!(:allow => "169.254.169.254") describe 'aodh::notifier' do diff --git a/aodh/spec/spec_helper.rb b/aodh/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/aodh/spec/spec_helper.rb +++ b/aodh/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/ceilometer/Gemfile b/ceilometer/Gemfile index fc2214398..984392186 100644 --- a/ceilometer/Gemfile +++ b/ceilometer/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/ceilometer/spec/classes/ceilometer_collector_spec.rb b/ceilometer/spec/classes/ceilometer_collector_spec.rb index b3bbfff80..c3e54c8ce 100644 --- a/ceilometer/spec/classes/ceilometer_collector_spec.rb +++ b/ceilometer/spec/classes/ceilometer_collector_spec.rb @@ -1,6 +1,4 @@ require 'spec_helper' -# LP1492636 - Cohabitation of compile matcher and webmock -WebMock.disable_net_connect!(:allow => "169.254.169.254") describe 'ceilometer::collector' do diff --git a/ceilometer/spec/spec_helper.rb b/ceilometer/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/ceilometer/spec/spec_helper.rb +++ b/ceilometer/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/cinder/Gemfile b/cinder/Gemfile index fc2214398..984392186 100644 --- a/cinder/Gemfile +++ b/cinder/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/cinder/spec/spec_helper.rb b/cinder/spec/spec_helper.rb index 9bc7bcf96..d3d363ea5 100644 --- a/cinder/spec/spec_helper.rb +++ b/cinder/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/glance/Gemfile b/glance/Gemfile index fc2214398..984392186 100644 --- a/glance/Gemfile +++ b/glance/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/glance/lib/puppet/provider/glance_swift_config/ini_setting.rb b/glance/lib/puppet/provider/glance_swift_config/ini_setting.rb new file mode 100644 index 000000000..9d2d7ece7 --- /dev/null +++ b/glance/lib/puppet/provider/glance_swift_config/ini_setting.rb @@ -0,0 +1,10 @@ +Puppet::Type.type(:glance_swift_config).provide( + :ini_setting, + :parent => Puppet::Type.type(:openstack_config).provider(:ini_setting) +) do + + def self.file_path + '/etc/glance/glance-swift.conf' + end + +end diff --git a/glance/lib/puppet/type/glance_swift_config.rb b/glance/lib/puppet/type/glance_swift_config.rb new file mode 100644 index 000000000..592b22f78 --- /dev/null +++ b/glance/lib/puppet/type/glance_swift_config.rb @@ -0,0 +1,57 @@ +Puppet::Type.newtype(:glance_swift_config) do + + ensurable + + newparam(:name, :namevar => true) do + desc 'Section/setting name to manage from glance-api.conf' + newvalues(/\S+\/\S+/) + end + + newproperty(:value) do + desc 'The value of the setting to be defined.' + munge do |value| + value = value.to_s.strip + value.capitalize! if value =~ /^(true|false)$/i + value + end + newvalues(/^[\S ]*$/) + + def is_to_s( currentvalue ) + if resource.secret? + return '[old secret redacted]' + else + return currentvalue + end + end + + def should_to_s( newvalue ) + if resource.secret? + return '[new secret redacted]' + else + return newvalue + end + end + end + + newparam(:secret, :boolean => true) do + desc 'Whether to hide the value from Puppet logs. Defaults to `false`.' + + newvalues(:true, :false) + + defaultto false + end + + newparam(:ensure_absent_val) do + desc 'A value that is specified as the value property will behave as if ensure => absent was specified' + defaultto('') + end + + autorequire(:package) do + if Facter.value(:osfamily) == 'Debian' + 'glance-api' + elsif Facter.value(:osfamily) == 'RedHat' + 'openstack-glance' + end + end + +end diff --git a/glance/manifests/backend/swift.pp b/glance/manifests/backend/swift.pp index 3a1adda4c..72750be7f 100644 --- a/glance/manifests/backend/swift.pp +++ b/glance/manifests/backend/swift.pp @@ -62,8 +62,11 @@ # == class: glance::backend::swift 'glance_store/swift_store_endpoint_type': value => $swift_store_endpoint_type; - 'glance_store/swift_store_config_file': value => '/etc/glance/glance-api.conf'; + 'glance_store/swift_store_config_file': value => '/etc/glance/glance-swift.conf'; 'glance_store/default_swift_reference': value => $default_swift_reference; + } + + glance_swift_config { "${default_swift_reference}/user": value => $swift_store_user; "${default_swift_reference}/key": value => $swift_store_key; "${default_swift_reference}/auth_address": value => $swift_store_auth_address; diff --git a/glance/spec/classes/glance_backend_swift_spec.rb b/glance/spec/classes/glance_backend_swift_spec.rb index 7ca3fa32c..d4531b30a 100644 --- a/glance/spec/classes/glance_backend_swift_spec.rb +++ b/glance/spec/classes/glance_backend_swift_spec.rb @@ -27,12 +27,12 @@ is_expected.to contain_glance_api_config('glance_store/swift_store_create_container_on_put').with_value(false) is_expected.to contain_glance_api_config('glance_store/swift_store_endpoint_type').with_value('internalURL') is_expected.to contain_glance_api_config('glance_store/swift_store_region').with_value(nil) - is_expected.to contain_glance_api_config('glance_store/swift_store_config_file').with_value('/etc/glance/glance-api.conf') + is_expected.to contain_glance_api_config('glance_store/swift_store_config_file').with_value('/etc/glance/glance-swift.conf') is_expected.to contain_glance_api_config('glance_store/default_swift_reference').with_value('ref1') - is_expected.to contain_glance_api_config('ref1/key').with_value('key') - is_expected.to contain_glance_api_config('ref1/user').with_value('user') - is_expected.to contain_glance_api_config('ref1/auth_version').with_value('2') - is_expected.to contain_glance_api_config('ref1/auth_address').with_value('127.0.0.1:5000/v2.0/') + is_expected.to contain_glance_swift_config('ref1/key').with_value('key') + is_expected.to contain_glance_swift_config('ref1/user').with_value('user') + is_expected.to contain_glance_swift_config('ref1/auth_version').with_value('2') + is_expected.to contain_glance_swift_config('ref1/auth_address').with_value('127.0.0.1:5000/v2.0/') end end @@ -60,10 +60,10 @@ is_expected.to contain_glance_api_config('glance_store/swift_store_endpoint_type').with_value('publicURL') is_expected.to contain_glance_api_config('glance_store/swift_store_region').with_value('RegionTwo') is_expected.to contain_glance_api_config('glance_store/default_swift_reference').with_value('swift_creds') - is_expected.to contain_glance_api_config('swift_creds/key').with_value('key2') - is_expected.to contain_glance_api_config('swift_creds/user').with_value('user2') - is_expected.to contain_glance_api_config('swift_creds/auth_version').with_value('1') - is_expected.to contain_glance_api_config('swift_creds/auth_address').with_value('127.0.0.2:8080/v1.0/') + is_expected.to contain_glance_swift_config('swift_creds/key').with_value('key2') + is_expected.to contain_glance_swift_config('swift_creds/user').with_value('user2') + is_expected.to contain_glance_swift_config('swift_creds/auth_version').with_value('1') + is_expected.to contain_glance_swift_config('swift_creds/auth_address').with_value('127.0.0.2:8080/v1.0/') end end diff --git a/glance/spec/spec_helper.rb b/glance/spec/spec_helper.rb index 144a79693..65f9fdb37 100644 --- a/glance/spec/spec_helper.rb +++ b/glance/spec/spec_helper.rb @@ -2,7 +2,6 @@ $LOAD_PATH.push(File.join(File.dirname(__FILE__), 'fixtures', 'modules', 'openstacklib', 'lib')) require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/glance/spec/unit/provider/glance_swift_config/ini_setting_spec.rb b/glance/spec/unit/provider/glance_swift_config/ini_setting_spec.rb new file mode 100644 index 000000000..90cd0cebf --- /dev/null +++ b/glance/spec/unit/provider/glance_swift_config/ini_setting_spec.rb @@ -0,0 +1,60 @@ +$LOAD_PATH.push( + File.join( + File.dirname(__FILE__), + '..', + '..', + '..', + 'fixtures', + 'modules', + 'openstacklib', + 'lib') +) + +require 'spec_helper' + +provider_class = Puppet::Type.type(:glance_swift_config).provider(:ini_setting) + +describe provider_class do + + it 'should default to the default setting when no other one is specified' do + resource = Puppet::Type::Glance_swift_config.new( + { + :name => 'DEFAULT/foo', + :value => 'bar' + } + ) + provider = provider_class.new(resource) + expect(provider.section).to eq('DEFAULT') + expect(provider.setting).to eq('foo') + end + + it 'should allow setting to be set explicitly' do + resource = Puppet::Type::Glance_swift_config.new( + { + :name => 'dude/foo', + :value => 'bar' + } + ) + provider = provider_class.new(resource) + expect(provider.section).to eq('dude') + expect(provider.setting).to eq('foo') + end + + it 'should ensure absent when is specified as a value' do + resource = Puppet::Type::Glance_swift_config.new( + {:name => 'dude/foo', :value => ''} + ) + provider = provider_class.new(resource) + provider.exists? + expect(resource[:ensure]).to eq :absent + end + + it 'should ensure absent when value matches ensure_absent_val' do + resource = Puppet::Type::Glance_swift_config.new( + {:name => 'dude/foo', :value => 'foo', :ensure_absent_val => 'foo' } + ) + provider = provider_class.new(resource) + provider.exists? + expect(resource[:ensure]).to eq :absent + end +end diff --git a/glance/spec/unit/type/glance_swift_config_spec.rb b/glance/spec/unit/type/glance_swift_config_spec.rb new file mode 100644 index 000000000..f72b89666 --- /dev/null +++ b/glance/spec/unit/type/glance_swift_config_spec.rb @@ -0,0 +1,41 @@ +require 'puppet' +require 'puppet/type/glance_swift_config' + +describe 'Puppet::Type.type(:glance_swift_config)' do + before :each do + Puppet::Type.rmtype(:glance_swift_config) + Facter.fact(:osfamily).stubs(:value).returns(platform_params[:osfamily]) + @glance_swift_config = Puppet::Type.type(:glance_swift_config).new(:name => 'DEFAULT/foo', :value => 'bar') + end + + shared_examples_for 'glance_swift_config' do + it 'should autorequire the package that install the file' do + catalog = Puppet::Resource::Catalog.new + package = Puppet::Type.type(:package).new(:name => platform_params[:package_name]) + catalog.add_resource package, @glance_swift_config + dependency = @glance_swift_config.autorequire + expect(dependency.size).to eq(1) + expect(dependency[0].target).to eq(@glance_swift_config) + expect(dependency[0].source).to eq(package) + end + end + + context 'on Debian platforms' do + let :platform_params do + { :package_name => 'glance-api', + :osfamily => 'Debian' } + end + + it_behaves_like 'glance_swift_config' + end + + context 'on RedHat platforms' do + let :platform_params do + { :package_name => 'openstack-glance', + :osfamily => 'RedHat'} + end + + it_behaves_like 'glance_swift_config' + end + +end diff --git a/heat/Gemfile b/heat/Gemfile index fc2214398..984392186 100644 --- a/heat/Gemfile +++ b/heat/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/heat/spec/spec_helper.rb b/heat/spec/spec_helper.rb index 9bc7bcf96..d3d363ea5 100644 --- a/heat/spec/spec_helper.rb +++ b/heat/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/horizon/Gemfile b/horizon/Gemfile index fc2214398..984392186 100644 --- a/horizon/Gemfile +++ b/horizon/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/horizon/spec/spec_helper.rb b/horizon/spec/spec_helper.rb index b3ef99e93..d096af0db 100644 --- a/horizon/spec/spec_helper.rb +++ b/horizon/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/keystone/lib/puppet/provider/keystone.rb b/keystone/lib/puppet/provider/keystone.rb index 124735890..1ba5f5cb9 100644 --- a/keystone/lib/puppet/provider/keystone.rb +++ b/keystone/lib/puppet/provider/keystone.rb @@ -29,7 +29,12 @@ def self.clean_host(host) when '::0' return '[::1]' else - return host + # if ipv6, make sure ip address has brackets - LP#1541512 + if host.include?(':') and !host.include?(']') + return "[" + host + "]" + else + return host + end end end diff --git a/keystone/manifests/params.pp b/keystone/manifests/params.pp index 0c8cc9b2c..d8c473702 100644 --- a/keystone/manifests/params.pp +++ b/keystone/manifests/params.pp @@ -32,7 +32,7 @@ $service_provider = undef $keystone_wsgi_script_source = '/usr/share/keystone/keystone.wsgi' $paste_config = '/usr/share/keystone/keystone-dist-paste.ini' - $pymysql_package_name = 'python2-PyMySQL' + $pymysql_package_name = undef } } } diff --git a/keystone/spec/classes/keystone_db_spec.rb b/keystone/spec/classes/keystone_db_spec.rb index 1918c95d8..2ff4213ed 100644 --- a/keystone/spec/classes/keystone_db_spec.rb +++ b/keystone/spec/classes/keystone_db_spec.rb @@ -34,8 +34,6 @@ it { is_expected.to contain_keystone_config('database/max_pool_size').with_value('21') } it { is_expected.to contain_keystone_config('database/max_overflow').with_value('21') } it { is_expected.to contain_keystone_config('database/retry_interval').with_value('11') } - it { is_expected.to contain_package('keystone-backend-package').with({ :ensure => 'present', :name => platform_params[:pymysql_package_name] }) } - end context 'with MySQL-python library as backend package' do @@ -83,11 +81,21 @@ } end - let :platform_params do - { :pymysql_package_name => 'python-pymysql' } - end - it_configures 'keystone::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://keystone:keystone@localhost/keystone', } + end + + it 'install the proper backend package' do + is_expected.to contain_package('keystone-backend-package').with( + :ensure => 'present', + :name => 'python-pymysql', + :tag => 'openstack' + ) + end + end end context 'on Redhat platforms' do @@ -97,11 +105,14 @@ } end - let :platform_params do - { :pymysql_package_name => 'python2-PyMySQL' } - end - it_configures 'keystone::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://keystone:keystone@localhost/keystone', } + end + it { is_expected.not_to contain_package('keystone-backend-package') } + end end end diff --git a/keystone/spec/unit/provider/keystone_spec.rb b/keystone/spec/unit/provider/keystone_spec.rb index 683e185e2..60afe4237 100644 --- a/keystone/spec/unit/provider/keystone_spec.rb +++ b/keystone/spec/unit/provider/keystone_spec.rb @@ -180,6 +180,14 @@ class AnotherKlass < Puppet::Provider::Keystone expect(klass.get_admin_endpoint).to eq('http://[::1]:5001') end + it 'should use [2620:52:0:23a9::25] in the admin endpoint if bind_host is 2620:52:0:23a9::25' do + mock = {'DEFAULT' => { 'admin_bind_host' => '2620:52:0:23a9::25', 'admin_port' => '5001' }} + File.expects(:exists?).with("/etc/keystone/keystone.conf").returns(true) + Puppet::Util::IniConfig::File.expects(:new).returns(mock) + mock.expects(:read).with('/etc/keystone/keystone.conf') + expect(klass.get_admin_endpoint).to eq('http://[2620:52:0:23a9::25]:5001') + end + it 'should use localhost in the admin endpoint if bind_host is unspecified' do mock = {'DEFAULT' => { 'admin_port' => '5001' }} File.expects(:exists?).with("/etc/keystone/keystone.conf").returns(true) diff --git a/manila/Gemfile b/manila/Gemfile index fc2214398..984392186 100644 --- a/manila/Gemfile +++ b/manila/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/manila/spec/spec_helper.rb b/manila/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/manila/spec/spec_helper.rb +++ b/manila/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/neutron/Gemfile b/neutron/Gemfile index fc2214398..984392186 100644 --- a/neutron/Gemfile +++ b/neutron/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/neutron/lib/puppet/provider/neutron.rb b/neutron/lib/puppet/provider/neutron.rb index 049d31808..135165859 100644 --- a/neutron/lib/puppet/provider/neutron.rb +++ b/neutron/lib/puppet/provider/neutron.rb @@ -1,5 +1,4 @@ require 'csv' -require 'json' require 'puppet/util/inifile' class Puppet::Provider::Neutron < Puppet::Provider @@ -132,91 +131,64 @@ def self.reset @neutron_credentials = nil end - def self.find_and_parse_json(text) - # separate json from any possible garbage around it and parse - rv = [] - if text.is_a? String - text = text.split("\n") - elsif !text.is_a? Array - return rv - end - found = false - (0..text.size-1).reverse_each do |line_no| - if text[line_no] =~ /\]\s*$/ - end_of_json_line_no = line_no - (0..end_of_json_line_no).reverse_each do |start_of_json_line_no| - if text[start_of_json_line_no] =~ /^\s*\[/ - begin - js_txt = text[start_of_json_line_no..end_of_json_line_no].join('') - rv = JSON.parse(js_txt) - found = true - rescue - # do nothing, next iteration please, because found==false - end - end - break if found - end - end - break if found - end - rv - end - def self.list_neutron_resources(type) ids = [] - list = auth_neutron("#{type}-list", '--format=json', - '--column=id') + list = cleanup_csv_with_id(auth_neutron("#{type}-list", '--format=csv', + '--column=id', '--quote=none')) if list.nil? raise(Puppet::ExecutionFailure, "Can't retrieve #{type}-list because Neutron or Keystone API is not available.") end - self.find_and_parse_json(list).each do |line| - ids << line['id'] + (list.split("\n")[1..-1] || []).compact.collect do |line| + ids << line.strip end - return ids end def self.get_neutron_resource_attrs(type, id) attrs = {} - net = auth_neutron("#{type}-show", '--format=json', id) + net = auth_neutron("#{type}-show", '--format=shell', id) if net.nil? raise(Puppet::ExecutionFailure, "Can't retrieve #{type}-show because Neutron or Keystone API is not available.") end - self.find_and_parse_json(net).each do |line| - k = line['Field'] - v = line['Value'] - if ['True', 'False'].include? v.to_s.capitalize - v = "#{v}".capitalize - elsif v.is_a? String and v =~ /\n/ - v = v.split(/\n/) - elsif v.is_a? Numeric - v = "#{v}" + last_key = nil + (net.split("\n") || []).compact.collect do |line| + if line.include? '=' + k, v = line.split('=', 2) + attrs[k] = v.gsub(/\A"|"\Z/, '') + last_key = k else - v = "#{v}" + # Handle the case of a list of values + v = line.gsub(/\A"|"\Z/, '') + attrs[last_key] = [attrs[last_key], v].flatten end - attrs[k] = v end - return attrs end def self.list_router_ports(router_name_or_id) results = [] cmd_output = auth_neutron("router-port-list", - '--format=json', + '--format=csv', router_name_or_id) + if ! cmd_output + return results + end - self.find_and_parse_json(cmd_output).each do |port| - if port['fixed_ips'] - fixed_ips = JSON.parse(port['fixed_ips']) - port['subnet_id'] = fixed_ips['subnet_id'] - port.delete('fixed_ips') + headers = nil + CSV.parse(cleanup_csv(cmd_output)) do |row| + if headers == nil + headers = row + else + result = Hash[*headers.zip(row).flatten] + match_data = /.*"subnet_id": "(.*)", .*/.match(result['fixed_ips']) + if match_data + result['subnet_id'] = match_data[1] + end + results << result end - results << port end - return results end @@ -250,4 +222,16 @@ def self.parse_creation_output(data) hash end + def self.cleanup_csv(text) + # Ignore warnings - assume legitimate output starts with a double quoted + # string. Errors will be caught and raised prior to this + text = text.split("\n").drop_while { |line| line !~ /^\".*\"/ }.join("\n") + "#{text}\n" + end + + def self.cleanup_csv_with_id(text) + return nil if text.nil? + text = text.split("\n").drop_while { |line| line !~ /^\s*id$/ }.join("\n") + "#{text}\n" + end end diff --git a/neutron/lib/puppet/type/neutron_plugin_ml2.rb b/neutron/lib/puppet/type/neutron_plugin_ml2.rb index 0ac513739..fd2da376f 100644 --- a/neutron/lib/puppet/type/neutron_plugin_ml2.rb +++ b/neutron/lib/puppet/type/neutron_plugin_ml2.rb @@ -14,6 +14,28 @@ value.capitalize! if value =~ /^(true|false)$/i value end + + def is_to_s( currentvalue ) + if resource.secret? + return '[old secret redacted]' + else + return currentvalue + end + end + + def should_to_s( newvalue ) + if resource.secret? + return '[new secret redacted]' + else + return newvalue + end + end + end + + newparam(:secret, :boolean => true) do + desc 'Whether to hide the value from Puppet logs. Defaults to `false`.' + newvalues(:true, :false) + defaultto false end newparam(:ensure_absent_val) do diff --git a/neutron/manifests/agents/bigswitch.pp b/neutron/manifests/agents/bigswitch.pp new file mode 100644 index 000000000..6b98a5d28 --- /dev/null +++ b/neutron/manifests/agents/bigswitch.pp @@ -0,0 +1,71 @@ +# == Class: neutron::agents::bigswitch +# +# Installs and configures the Big Switch agent and lldp +# +# === Parameters +# +# [*package_ensure*] +# (optional) The state of the package +# Defaults to present +# +# [*lldp_enabled*] +# (optional) The state of the neutron-bsn-lldp service +# Defaults to true +# +# [*agent_enabled*] +# (optional) The state of the neutron-bsn-agent service +# Defaults to false +# +# +class neutron::agents::bigswitch ( + $package_ensure = 'present', + $lldp_enabled = true, + $agent_enabled = false, +) { + + if($::osfamily != 'Redhat') { + fail("Unsupported osfamily ${::osfamily}") + } + + require ::neutron::plugins::ml2::bigswitch + + package { 'bigswitch-lldp': + ensure => $package_ensure, + name => $::neutron::params::bigswitch_lldp_package, + tag => 'openstack', + } + + package { 'bigswitch-agent': + ensure => $package_ensure, + name => $::neutron::params::bigswitch_agent_package, + tag => 'openstack', + } + + if $lldp_enabled { + $lldp_service_ensure = 'running' + } else { + $lldp_service_ensure = 'stopped' + } + + if $agent_enabled { + $agent_service_ensure = 'running' + } else { + $agent_service_ensure = 'stopped' + } + + service { 'bigswitch-lldp': + ensure => $lldp_service_ensure, + name => $::neutron::params::bigswitch_lldp_service, + enable => $lldp_enabled, + require => [Class['neutron'], Package['bigswitch-lldp']], + tag => 'neutron-service', + } + + service { 'bigswitch-agent': + ensure => $agent_service_ensure, + name => $::neutron::params::bigswitch_agent_service, + enable => $agent_enabled, + require => [Class['neutron'], Package['bigswitch-agent']], + tag => 'neutron-service', + } +} diff --git a/neutron/manifests/params.pp b/neutron/manifests/params.pp index 7f7f9a146..debecdcbf 100644 --- a/neutron/manifests/params.pp +++ b/neutron/manifests/params.pp @@ -26,6 +26,11 @@ $sriov_nic_agent_service = 'neutron-sriov-nic-agent' $sriov_nic_agent_package = 'openstack-neutron-sriov-nic-agent' + $bigswitch_lldp_package = 'openstack-neutron-bigswitch-lldp' + $bigswitch_agent_package = 'openstack-neutron-bigswitch-agent' + $bigswitch_lldp_service = 'neutron-bsn-lldp' + $bigswitch_agent_service = 'neutron-bsn-agent' + $cisco_server_package = 'openstack-neutron-cisco' $cisco_config_file = '/etc/neutron/plugins/cisco/cisco_plugins.ini' # Add templated Cisco Nexus ML2 config to confdir diff --git a/neutron/manifests/plugins/ml2/bigswitch/restproxy.pp b/neutron/manifests/plugins/ml2/bigswitch/restproxy.pp index a53ab769b..4684394f2 100644 --- a/neutron/manifests/plugins/ml2/bigswitch/restproxy.pp +++ b/neutron/manifests/plugins/ml2/bigswitch/restproxy.pp @@ -33,6 +33,22 @@ # (optional) Directory where Big Switch controller certificate will be # stored. Defaults to '/var/lib/neutron'. # +# [*auth_tenant*] +# (optional) The tenant of the auth user +# Defaults to service +# +# [*auth_password*] +# (optional) The password to use for authentication (keystone) +# Defaults to false. +# +# [*auth_user*] +# (optional) The name of the auth user +# Defaults to neutron +# +# [*auth_url*] +# (optional) Complete public Identity API endpoint. +# Defaults to: false +# class neutron::plugins::ml2::bigswitch::restproxy ( $servers, $server_auth, @@ -42,6 +58,11 @@ $neutron_id = 'neutron', $server_ssl = true, $ssl_cert_directory = '/var/lib/neutron', + + $auth_tenant = 'service', + $auth_password = false, + $auth_user = 'neutron', + $auth_url = false, ) { require ::neutron::plugins::ml2::bigswitch @@ -54,5 +75,10 @@ 'restproxy/neutron_id' : value => $neutron_id; 'restproxy/server_ssl' : value => $server_ssl; 'restproxy/ssl_cert_directory' : value => $ssl_cert_directory; + + 'restproxy/auth_tenant' : value => $auth_tenant; + 'restproxy/auth_password' : value => $auth_password, secret => true; + 'restproxy/auth_user' : value => $auth_user; + 'restproxy/auth_url' : value => $auth_url; } } diff --git a/neutron/spec/classes/neutron_agents_bigswitch.rb b/neutron/spec/classes/neutron_agents_bigswitch.rb new file mode 100644 index 000000000..67318d1d4 --- /dev/null +++ b/neutron/spec/classes/neutron_agents_bigswitch.rb @@ -0,0 +1,114 @@ +require 'spec_helper' + +describe 'neutron::agents::bigswitch' do + + let :pre_condition do + "class { 'neutron': rabbit_password => 'passw0rd' }" + end + + let :test_facts do + { + :operatingsystem => 'default', + :operatingsystemrelease => 'default', + :package_ensure => 'present', + } + end + + shared_examples_for 'neutron bigswitch base' do + it 'should have' do + is_expected.to contain_package('python-networking-bigswitch').with( + :ensure => 'present', + :tag => 'openstack' + ) + end + end + + context 'neutron-bsn-agent only' do + let :facts do + @default_facts.merge(test_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + })) + end + + let :params do + { + :lldp_enabled => false, + :agent_enabled => true + } + end + + it_configures 'neutron bigswitch base' + + it 'enable neutron-bsn-agent service' do + is_expected.to contain_service('bigswitch-agent').with( + :enable => params[:agent_enabled], + :ensure =>'running', + :tag =>'neutron-service', + ) + end + + it 'disable neutron-bsn-lldp service' do + is_expected.to contain_service('bigswitch-lldp').with( + :enable => params[:lldp_enabled], + :ensure =>'stopped', + :tag =>'neutron-service', + ) + end + + end + + context 'neutron-bsn-lldp only' do + let :facts do + @default_facts.merge(test_facts.merge({ + :osfamily => 'RedHat', + :operatingsystemrelease => '7' + })) + end + + let :params do + { + :lldp_enabled => true, + :agent_enabled => false + } + end + + it_configures 'neutron bigswitch base' + + it 'disable neutron-bsn-agent service' do + is_expected.to contain_service('bigswitch-agent').with( + :enable => params[:agent_enabled], + :ensure =>'stopped', + :tag =>'neutron-service', + ) + end + + it 'enable neutron-bsn-lldp service' do + is_expected.to contain_service('bigswitch-lldp').with( + :enable => params[:lldp_enabled], + :ensure =>'running', + :tag =>'neutron-service', + ) + end + + end + + context 'on Debian platforms' do + let :facts do + @default_facts.merge(test_facts.merge({ + :osfamily => 'Debian' + })) + end + + let :params do + { + :lldp_enabled => false, + :agent_enabled => false + } + end + + it { is_expected.to raise_error(Puppet::Error, /Unsupported osfamily Debian/) } + + end + +end diff --git a/neutron/spec/classes/neutron_plugins_ml2_bigswitch_restproxy_spec.rb b/neutron/spec/classes/neutron_plugins_ml2_bigswitch_restproxy_spec.rb index a21564086..9927be96b 100644 --- a/neutron/spec/classes/neutron_plugins_ml2_bigswitch_restproxy_spec.rb +++ b/neutron/spec/classes/neutron_plugins_ml2_bigswitch_restproxy_spec.rb @@ -44,6 +44,11 @@ class { 'neutron': is_expected.to contain_neutron_plugin_ml2('restproxy/neutron_id').with_value('neutron') is_expected.to contain_neutron_plugin_ml2('restproxy/server_ssl').with_value(true) is_expected.to contain_neutron_plugin_ml2('restproxy/ssl_cert_directory').with_value('/var/lib/neutron') + + is_expected.to contain_neutron_plugin_ml2('restproxy/auth_tenant').with_value('service') + is_expected.to contain_neutron_plugin_ml2('restproxy/auth_password').with_value(false) + is_expected.to contain_neutron_plugin_ml2('restproxy/auth_user').with_value('neutron') + is_expected.to contain_neutron_plugin_ml2('restproxy/auth_url').with_value(false) end context 'with custom params' do @@ -63,6 +68,11 @@ class { 'neutron': is_expected.to contain_neutron_plugin_ml2('restproxy/neutron_id').with_value('openstack') is_expected.to contain_neutron_plugin_ml2('restproxy/server_ssl').with_value(false) is_expected.to contain_neutron_plugin_ml2('restproxy/ssl_cert_directory').with_value('/var/lib/bigswitch') + + is_expected.to contain_neutron_plugin_ml2('restproxy/auth_tenant').with_value('service') + is_expected.to contain_neutron_plugin_ml2('restproxy/auth_password').with_value(false) + is_expected.to contain_neutron_plugin_ml2('restproxy/auth_user').with_value('neutron') + is_expected.to contain_neutron_plugin_ml2('restproxy/auth_url').with_value(false) end end diff --git a/neutron/spec/spec_helper.rb b/neutron/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/neutron/spec/spec_helper.rb +++ b/neutron/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/neutron/spec/unit/provider/neutron_router/neutron_spec.rb b/neutron/spec/unit/provider/neutron_router/neutron_spec.rb index c6e960d5c..9016ab007 100644 --- a/neutron/spec/unit/provider/neutron_router/neutron_spec.rb +++ b/neutron/spec/unit/provider/neutron_router/neutron_spec.rb @@ -87,25 +87,15 @@ end it 'should detect a gateway net id' do - output = <<-EOT - bla-bla-bla - - [{"Field": "external_gateway_info", - "Value": "{\\"network_id\\": \\"1b-b1\\", \\"enable_snat\\": true, \\"external_fixed_ips\\": [{\\"subnet_id\\": \\"1b-b1\\", \\"ip_address\\": \\"1.1.1.1\\"}]}" - }] - EOT - klass.stubs(:auth_neutron).returns(output) + klass.stubs(:auth_neutron).returns( + 'external_gateway_info="{\"network_id\": \"1b-b1\", \"enable_snat\": true, \"external_fixed_ips\": [{\"subnet_id\": \"1b-b1\", \"ip_address\": \"1.1.1.1\"}]}"' + ) result = klass.get_neutron_resource_attrs 'foo', nil expect(provider.parse_gateway_network_id(result['external_gateway_info'])).to eql('1b-b1') end it 'should return empty value, if there is no net id found' do - output = <<-EOT - bla-bla-bla - - [{"Field": "external_gateway_info", "Value": "{}"}] - EOT - klass.stubs(:auth_neutron).returns(output) + klass.stubs(:auth_neutron).returns('external_gateway_info="{}"') result = klass.get_neutron_resource_attrs 'foo', nil expect(provider.parse_gateway_network_id(result['external_gateway_info'])).to eql('') end diff --git a/neutron/spec/unit/provider/neutron_spec.rb b/neutron/spec/unit/provider/neutron_spec.rb index 98cc5b082..3b767005c 100644 --- a/neutron/spec/unit/provider/neutron_spec.rb +++ b/neutron/spec/unit/provider/neutron_spec.rb @@ -131,9 +131,9 @@ def klass it 'should exclude the column header' do output = <<-EOT - bla-bla-bla - - [{"id": "net1"},{"id": "net2"}] +id +net1 +net2 EOT klass.expects(:auth_neutron).returns(output) result = klass.list_neutron_resources('foo') @@ -141,19 +141,6 @@ def klass end it 'should return empty list when there are no neutron resources' do - output = <<-EOT - bla-bla-bla - - [] - - bla-bla - EOT - klass.stubs(:auth_neutron).returns(output) - result = klass.list_neutron_resources('foo') - expect(result).to eql([]) - end - - it 'should return empty respons when there are no neutron resources' do output = <<-EOT EOT klass.stubs(:auth_neutron).returns(output) @@ -173,21 +160,16 @@ def klass describe 'when retrieving attributes for neutron resources' do it 'should parse single-valued attributes into a key-value pair' do - output = <<-EOT - bla-bla-bla - - [{"Field": "admin_state_up", "Value": true}] - EOT - klass.expects(:auth_neutron).returns(output) + klass.expects(:auth_neutron).returns('admin_state_up="True"') result = klass.get_neutron_resource_attrs('foo', 'id') - expect(result).to eql({"admin_state_up" => "True"}) + expect(result).to eql({"admin_state_up" => 'True'}) end it 'should parse multi-valued attributes into a key-list pair' do output = <<-EOT - bla-bla-bla - - [{"Field": "subnets", "Value": "subnet1\\nsubnet2\\nsubnet3"}] +subnets="subnet1 +subnet2 +subnet3" EOT klass.expects(:auth_neutron).returns(output) result = klass.get_neutron_resource_attrs('foo', 'id') @@ -204,40 +186,35 @@ def klass it 'should handle an empty port list' do klass.expects(:auth_neutron).with('router-port-list', - '--format=json', + '--format=csv', router) result = klass.list_router_ports(router) expect(result).to eql([]) end it 'should handle several ports' do - output = ''' - [ - { - "id": "1345e576-a21f-4c2e-b24a-b245639852ab", - "name": "", - "mac_address": "fa:16:3e:e3:e6:38", - "fixed_ips": "{\"subnet_id\": \"839a1d2d-2c6e-44fb-9a2b-9b011dce8c2f\", \"ip_address\": \"10.0.0.1\"}" - }, - { - "id": "de0dc526-02b2-467c-9832-2c3dc69ac2b4", - "name": "", - "mac_address": "fa:16:3e:f6:b5:72", - "fixed_ips": "{\"subnet_id\": \"e4db0abd-276a-4f69-92ea-8b9e4eacfd43\", \"ip_address\": \"172.24.4.226\"}" - } - ] - ''' + output = <<-EOT +"id","name","mac_address","fixed_ips" +"1345e576-a21f-4c2e-b24a-b245639852ab","","fa:16:3e:e3:e6:38","{""subnet_id"": ""839a1d2d-2c6e-44fb-9a2b-9b011dce8c2f"", ""ip_address"": ""10.0.0.1""}" +"de0dc526-02b2-467c-9832-2c3dc69ac2b4","","fa:16:3e:f6:b5:72","{""subnet_id"": ""e4db0abd-276a-4f69-92ea-8b9e4eacfd43"", ""ip_address"": ""172.24.4.226""}" + EOT expected = - [{ "name"=>"", + [{ "fixed_ips"=> + "{\"subnet_id\": \"839a1d2d-2c6e-44fb-9a2b-9b011dce8c2f\", \ +\"ip_address\": \"10.0.0.1\"}", + "name"=>"", "subnet_id"=>"839a1d2d-2c6e-44fb-9a2b-9b011dce8c2f", "id"=>"1345e576-a21f-4c2e-b24a-b245639852ab", "mac_address"=>"fa:16:3e:e3:e6:38"}, - { "name"=>"", + {"fixed_ips"=> + "{\"subnet_id\": \"e4db0abd-276a-4f69-92ea-8b9e4eacfd43\", \ +\"ip_address\": \"172.24.4.226\"}", + "name"=>"", "subnet_id"=>"e4db0abd-276a-4f69-92ea-8b9e4eacfd43", "id"=>"de0dc526-02b2-467c-9832-2c3dc69ac2b4", "mac_address"=>"fa:16:3e:f6:b5:72"}] klass.expects(:auth_neutron). - with('router-port-list', '--format=json', router). + with('router-port-list', '--format=csv', router). returns(output) result = klass.list_router_ports(router) expect(result).to eql(expected) @@ -266,67 +243,42 @@ def klass end - describe 'should parse valid json output, covered by garbage' do - - it 'should parse valid output into a list of hashes' do - data = ''' - /usr/lib/python2.7/dist-packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. - InsecurePlatformWarning - /usr/lib/python2.7/dist-packages/urllib3/connection.py:251: SecurityWarning: Certificate has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.) - SecurityWarning - [{"Field": "allocation_pools", "Value": "{\"start\": \"192.168.111.2\", \"end\": \"192.168.111.254\"}"}, {"Field": "cidr", "Value": "192.168.111.0/24"}, - {"Field": "dns_nameservers", "Value": "8.8.4.4\n8.8.8.8"}, {"Field": "enable_dhcp", "Value": true}, {"Field": "gateway_ip", "Value": "192.168.111.1"}, {"Field": "host_routes", "Value": ""}, {"Field": "id", "Value": "b87fbfd1-0e52-4ab6-8987-286ef0912d1f"}, {"Field": "ip_version", "Value": 4}, {"Field": "ipv6_address_mode", "Value": ""}, {"Field": "ipv6_ra_mode", "Value": ""}, - {"Field": "XXX", "Value": - [1, - 2,3]}, - {"Field": "name", "Value": "net04__subnet"}, {"Field": "network_id", "Value": "d70b399b-668b-4861-b092-4876ec65df60"}, {"Field": "subnetpool_id", "Value": ""}, {"Field": "tenant_id", "Value": "2764315d0ec24a07bf3773057aa51142"}] - xxx yyy zz - eof - ''' - expected = [ - {"Field"=>"allocation_pools", "Value"=>"{\"start\": \"192.168.111.2\", \"end\": \"192.168.111.254\"}"}, - {"Field"=>"cidr", "Value"=>"192.168.111.0/24"}, - {"Field"=>"dns_nameservers", "Value"=>"8.8.4.4\n8.8.8.8"}, - {"Field"=>"enable_dhcp", "Value"=>true}, - {"Field"=>"gateway_ip", "Value"=>"192.168.111.1"}, - {"Field"=>"host_routes", "Value"=>""}, - {"Field"=>"id", "Value"=>"b87fbfd1-0e52-4ab6-8987-286ef0912d1f"}, - {"Field"=>"ip_version", "Value"=>4}, - {"Field"=>"ipv6_address_mode", "Value"=>""}, - {"Field"=>"ipv6_ra_mode", "Value"=>""}, - {"Field"=>"XXX", "Value"=>[1, 2, 3]}, - {"Field"=>"name", "Value"=>"net04__subnet"}, - {"Field"=>"network_id", "Value"=>"d70b399b-668b-4861-b092-4876ec65df60"}, - {"Field"=>"subnetpool_id", "Value"=>""}, - {"Field"=>"tenant_id", "Value"=>"2764315d0ec24a07bf3773057aa51142"}] - expect(klass.find_and_parse_json(data)).to eq(expected) - end - end + describe 'garbage in the csv output' do + it '#list_router_ports' do + output = <<-EOT +/usr/lib/python2.7/dist-packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. + InsecurePlatformWarning +"id","name","mac_address","fixed_ips" +"1345e576-a21f-4c2e-b24a-b245639852ab","","fa:16:3e:e3:e6:38","{""subnet_id"": ""839a1d2d-2c6e-44fb-9a2b-9b011dce8c2f"", ""ip_address"": ""10.0.0.1""}" + EOT + expected = [{ "fixed_ips"=> + "{\"subnet_id\": \"839a1d2d-2c6e-44fb-9a2b-9b011dce8c2f\", \ +\"ip_address\": \"10.0.0.1\"}", + "name"=>"", + "subnet_id"=>"839a1d2d-2c6e-44fb-9a2b-9b011dce8c2f", + "id"=>"1345e576-a21f-4c2e-b24a-b245639852ab", + "mac_address"=>"fa:16:3e:e3:e6:38"}] - describe 'should parse valid json output, and convert booleans to idempotent strings' do - it 'boolean values should converted to capitalized strings' do - output = ''' - [{"Field": "allocation_pools", "Value": "{\"start\": \"192.168.111.2\", \"end\": \"192.168.111.254\"}"}, {"Field": "cidr", "Value": "192.168.111.0/24"}, - {"Field": "dns_nameservers", "Value": "8.8.4.4\n8.8.8.8"}, {"Field": "enable_dhcp", "Value": true}, {"Field": "gateway_ip", "Value": "192.168.111.1"}, {"Field": "host_routes", "Value": ""}, {"Field": "id", "Value": "b87fbfd1-0e52-4ab6-8987-286ef0912d1f"}, {"Field": "ip_version", "Value": 4}, {"Field": "ipv6_address_mode", "Value": ""}, {"Field": "ipv6_ra_mode", "Value": ""}, - {"Field": "YYY", "Value": false}] - ''' - klass.stubs(:auth_neutron).returns(output) - result = klass.get_neutron_resource_attrs 'foo', nil - expect(result['enable_dhcp']).to eql('True') - expect(result['YYY']).to eql('False') + klass.expects(:auth_neutron). + with('router-port-list', '--format=csv', 'router1'). + returns(output) + result = klass.list_router_ports('router1') + expect(result).to eql(expected) end - it 'stringifyed boolean values should converted to capitalized strings' do - output = ''' - [{"Field": "allocation_pools", "Value": "{\"start\": \"192.168.111.2\", \"end\": \"192.168.111.254\"}"}, {"Field": "cidr", "Value": "192.168.111.0/24"}, - {"Field": "dns_nameservers", "Value": "8.8.4.4\n8.8.8.8"}, {"Field": "enable_dhcp", "Value": "True"}, {"Field": "gateway_ip", "Value": "192.168.111.1"}, {"Field": "host_routes", "Value": ""}, {"Field": "id", "Value": "b87fbfd1-0e52-4ab6-8987-286ef0912d1f"}, {"Field": "ip_version", "Value": 4}, {"Field": "ipv6_address_mode", "Value": ""}, {"Field": "ipv6_ra_mode", "Value": ""}, - {"Field": "YYY", "Value": "false"}] - ''' - klass.stubs(:auth_neutron).returns(output) - result = klass.get_neutron_resource_attrs 'foo', nil - expect(result['enable_dhcp']).to eql('True') - expect(result['YYY']).to eql('False') + it '#list_neutron_resources' do + output = <<-EOT +/usr/lib/python2.7/dist-packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. + InsecurePlatformWarning +id +4a305398-d806-46c5-a6aa-dcd6a4a99330 + EOT + klass.expects(:auth_neutron). + with('subnet-list', '--format=csv', '--column=id', '--quote=none'). + returns(output) + expected = ['4a305398-d806-46c5-a6aa-dcd6a4a99330'] + result = klass.list_neutron_resources('subnet') + expect(result).to eql(expected) end end - end diff --git a/nova/Gemfile b/nova/Gemfile index c22e44167..03fd52161 100644 --- a/nova/Gemfile +++ b/nova/Gemfile @@ -13,14 +13,13 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' - gem 'r10k', :require => 'false' gem 'netaddr', :require => 'false' end group :system_tests do gem 'beaker-rspec', :require => 'false' gem 'beaker-puppet_install_helper', :require => 'false' + gem 'r10k', :require => 'false' end if facterversion = ENV['FACTER_GEM_VERSION'] diff --git a/nova/spec/classes/nova_vnc_proxy_spec.rb b/nova/spec/classes/nova_vnc_proxy_spec.rb index 302708850..8ac5430eb 100644 --- a/nova/spec/classes/nova_vnc_proxy_spec.rb +++ b/nova/spec/classes/nova_vnc_proxy_spec.rb @@ -1,6 +1,4 @@ require 'spec_helper' -# LP1492636 - Cohabitation of compile matcher and webmock -WebMock.disable_net_connect!(:allow => "169.254.169.254") describe 'nova::vncproxy' do diff --git a/nova/spec/spec_helper.rb b/nova/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/nova/spec/spec_helper.rb +++ b/nova/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/openstack_extras/Gemfile b/openstack_extras/Gemfile index fc2214398..984392186 100644 --- a/openstack_extras/Gemfile +++ b/openstack_extras/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/openstack_extras/spec/spec_helper.rb b/openstack_extras/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/openstack_extras/spec/spec_helper.rb +++ b/openstack_extras/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/openstacklib/Gemfile b/openstacklib/Gemfile index fc2214398..984392186 100644 --- a/openstacklib/Gemfile +++ b/openstacklib/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/openstacklib/spec/spec_helper.rb b/openstacklib/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/openstacklib/spec/spec_helper.rb +++ b/openstacklib/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/tempest/Gemfile b/tempest/Gemfile index fc2214398..984392186 100644 --- a/tempest/Gemfile +++ b/tempest/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/tempest/lib/puppet/provider/tempest.rb b/tempest/lib/puppet/provider/tempest.rb new file mode 100644 index 000000000..97f7cb619 --- /dev/null +++ b/tempest/lib/puppet/provider/tempest.rb @@ -0,0 +1,44 @@ +require 'puppet/util/inifile' +require 'puppet/provider/openstack' +require 'puppet/provider/openstack/auth' +require 'puppet/provider/openstack/credentials' +class Puppet::Provider::Tempest < Puppet::Provider::Openstack + + extend Puppet::Provider::Openstack::Auth + + def self.tempest_file + return @tempest_file if @tempest_file + @tempest_file = Puppet::Util::IniConfig::File.new + @tempest_file.read(@file_path) + @tempest_file + end + + def self.request(service, action, properties=[], file_path) + @file_path = file_path + begin + super(service, action, properties) + rescue Puppet::Error::OpenstackAuthInputError => error + tempest_request(service, action, error, properties) + end + end + + def self.tempest_request(service, action, error, properties=nil) + @credentials.username = tempest_credentials['admin_user'] + @credentials.password = tempest_credentials['admin_password'] + @credentials.project_name = tempest_credentials['admin_tenant_name'] + @credentials.auth_url = tempest_credentials['auth_endpoint'] + raise error unless @credentials.set? + Puppet::Provider::Openstack.request(service, action, properties, @credentials) + end + + def self.tempest_credentials + t = {} + t['admin_user'] = tempest_file['identity']['admin_username'] + t['admin_password'] = tempest_file['identity']['admin_password'] + t['admin_tenant_name'] = tempest_file['identity']['admin_tenant_name'] + t['auth_endpoint'] = tempest_file['identity']['uri'] + return t + end + + +end diff --git a/tempest/lib/puppet/provider/tempest_glance_id_setter/ruby.rb b/tempest/lib/puppet/provider/tempest_glance_id_setter/openstack.rb similarity index 53% rename from tempest/lib/puppet/provider/tempest_glance_id_setter/ruby.rb rename to tempest/lib/puppet/provider/tempest_glance_id_setter/openstack.rb index dc15915d8..bc43360e8 100644 --- a/tempest/lib/puppet/provider/tempest_glance_id_setter/ruby.rb +++ b/tempest/lib/puppet/provider/tempest_glance_id_setter/openstack.rb @@ -1,6 +1,11 @@ -Puppet::Type.type(:tempest_glance_id_setter).provide(:ruby) do +require File.join(File.dirname(__FILE__), '..','..','..', 'puppet/provider/tempest') - # almost entirely lifted from stdlib's file_line +Puppet::Type.type(:tempest_glance_id_setter).provide( + :openstack, + :parent => Puppet::Provider::Tempest +) do + + @credentials = Puppet::Provider::Openstack::CredentialsV2_0.new def exists? lines.find do |line| @@ -8,13 +13,32 @@ def exists? end end + def file_path + resource[:tempest_conf_path] + end + def create handle_create_with_match end + def destroy + handle_create_with_match + end + def get_image_id - @image_id ||= Puppet::Resource.indirection.find("Glance_image/#{resource[:image_name]}")[:id] - @image_id if @image_id != :absent + if resource[:ensure] == :present or resource[:ensure].nil? + if @image_id.nil? + images = self.class.request('image', 'list', file_path) + img = images.detect {|img| img[:name] == resource[:image_name]} + if img.nil? + raise(Puppet::Error, "Image #{resource[:image_name]} not found!") + end + @image_id = img[:id] + end + elsif resource[:ensure] != :absent + raise(Puppet::Error, "Cannot ensure to #{resource[:ensure]}") + end + @image_id end def should_line @@ -32,7 +56,7 @@ def handle_create_with_match() file = lines case match_count when 1 - File.open(resource[:tempest_conf_path], 'w') do |fh| + File.open(file_path, 'w') do |fh| lines.each do |l| fh.puts(regex.match(l) ? "#{should_line}" : l) end @@ -44,10 +68,10 @@ def handle_create_with_match() else file.insert(block_pos+1, "#{should_line}\n") end - File.write(resource[:tempest_conf_path], file.join) + File.write(file_path, file.join) else # cannot be negative. raise Puppet::Error, "More than one line in file \ -'#{resource[:tempest_conf_path]}' matches pattern '#{regex}'" +'#{file_path}' matches pattern '#{regex}'" end end @@ -58,7 +82,7 @@ def lines # file; for now assuming that this type is only used on # small-ish config files that can fit into memory without # too much trouble. - @lines ||= File.readlines(resource[:tempest_conf_path]) + @lines ||= File.readlines(file_path) end end diff --git a/tempest/lib/puppet/provider/tempest_neutron_net_id_setter/ruby.rb b/tempest/lib/puppet/provider/tempest_neutron_net_id_setter/openstack.rb similarity index 53% rename from tempest/lib/puppet/provider/tempest_neutron_net_id_setter/ruby.rb rename to tempest/lib/puppet/provider/tempest_neutron_net_id_setter/openstack.rb index 2c605d346..1a60133d5 100644 --- a/tempest/lib/puppet/provider/tempest_neutron_net_id_setter/ruby.rb +++ b/tempest/lib/puppet/provider/tempest_neutron_net_id_setter/openstack.rb @@ -1,6 +1,11 @@ -Puppet::Type.type(:tempest_neutron_net_id_setter).provide(:ruby) do +require File.join(File.dirname(__FILE__), '..','..','..', 'puppet/provider/tempest') - # almost entirely lifted from stdlib's file_line +Puppet::Type.type(:tempest_neutron_net_id_setter).provide( + :openstack, + :parent => Puppet::Provider::Tempest +) do + + @credentials = Puppet::Provider::Openstack::CredentialsV2_0.new def exists? lines.find do |line| @@ -8,13 +13,32 @@ def exists? end end + def file_path + resource[:tempest_conf_path] + end + def create handle_create_with_match end + def destroy + handle_create_with_match + end + def get_network_id - @network_id ||= Puppet::Resource.indirection.find("Neutron_network/#{@resource[:network_name]}")[:id] - @network_id if @network_id != :absent + if resource[:ensure] == :present or resource[:ensure].nil? + if @network_id.nil? + nets = self.class.request('network', 'list', file_path) + net = nets.detect {|img| img[:name] == resource[:network_name]} + if net.nil? + raise(Puppet::Error, "Network #{resource[:network_name]} not found!") + end + @network_id = net[:id] + end + elsif resource[:ensure] != :absent + raise(Puppet::Error, "Cannot ensure to #{resource[:ensure]}") + end + @network_id end def should_line @@ -31,7 +55,7 @@ def handle_create_with_match() file = lines case match_count when 1 - File.open(resource[:tempest_conf_path], 'w') do |fh| + File.open(file_path, 'w') do |fh| lines.each do |l| fh.puts(regex.match(l) ? should_line : l) end @@ -43,10 +67,10 @@ def handle_create_with_match() else file.insert(block_pos+1, "#{should_line}\n") end - File.write(resource[:tempest_conf_path], file.join) + File.write(file_path, file.join) else # cannot be negative. raise Puppet::Error, "More than one line in file \ -'#{resource[:tempest_conf_path]}' matches pattern '#{regex}'" +'#{file_path}' matches pattern '#{regex}'" end end @@ -57,7 +81,7 @@ def lines # file; for now assuming that this type is only used on # small-ish config files that can fit into memory without # too much trouble. - @lines ||= File.readlines(resource[:tempest_conf_path]) + @lines ||= File.readlines(file_path) end end diff --git a/tempest/manifests/init.pp b/tempest/manifests/init.pp index bbeeded02..bb2841e21 100644 --- a/tempest/manifests/init.pp +++ b/tempest/manifests/init.pp @@ -348,6 +348,7 @@ } Glance_image<||> -> Tempest_glance_id_setter['image_ref'] Tempest_config<||> -> Tempest_glance_id_setter['image_ref'] + Keystone_user_role<||> -> Tempest_glance_id_setter['image_ref'] } elsif ($image_name and $image_ref) or (! $image_name and ! $image_ref) { fail('A value for either image_name or image_ref must be provided.') } @@ -359,6 +360,7 @@ } Glance_image<||> -> Tempest_glance_id_setter['image_ref_alt'] Tempest_config<||> -> Tempest_glance_id_setter['image_ref_alt'] + Keystone_user_role<||> -> Tempest_glance_id_setter['image_ref_alt'] } elsif ($image_name_alt and $image_ref_alt) or (! $image_name_alt and ! $image_ref_alt) { fail('A value for either image_name_alt or image_ref_alt must \ be provided.') @@ -374,6 +376,7 @@ } Neutron_network<||> -> Tempest_neutron_net_id_setter['public_network_id'] Tempest_config<||> -> Tempest_neutron_net_id_setter['public_network_id'] + Keystone_user_role<||> -> Tempest_neutron_net_id_setter['public_network_id'] } elsif ($public_network_name and $public_network_id) or (! $public_network_name and ! $public_network_id) { fail('A value for either public_network_id or public_network_name \ must be provided.') diff --git a/tempest/metadata.json b/tempest/metadata.json index 33f95ac9b..54f518936 100644 --- a/tempest/metadata.json +++ b/tempest/metadata.json @@ -33,6 +33,7 @@ "dependencies": [ { "name": "puppetlabs/inifile", "version_requirement": ">=1.0.0 <2.0.0" }, { "name": "puppetlabs/stdlib", "version_requirement": ">=4.0.0 <5.0.0" }, - { "name": "puppetlabs/vcsrepo", "version_requirement": ">=0.1.2 <2.0.0"} + { "name": "puppetlabs/vcsrepo", "version_requirement": ">=0.1.2 <2.0.0"}, + { "name": "openstack/openstacklib", "version_requirement": ">= 7.0.0 <8.0.0" } ] } diff --git a/tempest/spec/spec_helper.rb b/tempest/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/tempest/spec/spec_helper.rb +++ b/tempest/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/tempest/spec/unit/puppet/provider/ruby_spec.rb b/tempest/spec/unit/puppet/provider/ruby_spec.rb index dbf8ae979..35444beca 100644 --- a/tempest/spec/unit/puppet/provider/ruby_spec.rb +++ b/tempest/spec/unit/puppet/provider/ruby_spec.rb @@ -3,9 +3,9 @@ describe 'Providers' do glance_provider_class = - Puppet::Type.type(:tempest_glance_id_setter).provider(:ruby) + Puppet::Type.type(:tempest_glance_id_setter).provider(:openstack) network_provider_class = - Puppet::Type.type(:tempest_neutron_net_id_setter).provider(:ruby) + Puppet::Type.type(:tempest_neutron_net_id_setter).provider(:openstack) include PuppetlabsSpec::Files let(:tmpfile) { tmpfilename('ini_setting_test') } diff --git a/tripleo/Gemfile b/tripleo/Gemfile index 91c5f0a3e..bdf11ff83 100644 --- a/tripleo/Gemfile +++ b/tripleo/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' # adding 'psych' explicitly # https://github.com/bundler/bundler/issues/2068 # TODO: drop it in a future release of 'bundle'. diff --git a/tripleo/manifests/loadbalancer.pp b/tripleo/manifests/loadbalancer.pp index f49ea1218..8c21ab7c4 100644 --- a/tripleo/manifests/loadbalancer.pp +++ b/tripleo/manifests/loadbalancer.pp @@ -690,6 +690,10 @@ haproxy::listen { 'keystone_public': bind => $keystone_public_bind_opts, collect_exported => false, + mode => 'http', # Needed for http-request option + options => { + 'http-request' => ['set-header X-Forwarded-Proto https if { ssl_fc }'], + }, } haproxy::balancermember { 'keystone_public': listening_service => 'keystone_public', diff --git a/tripleo/spec/spec_helper.rb b/tripleo/spec/spec_helper.rb index 5cf9642eb..15d5eab32 100644 --- a/tripleo/spec/spec_helper.rb +++ b/tripleo/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' fixture_path = File.expand_path(File.join(__FILE__, '..', 'fixtures')) diff --git a/trove/Gemfile b/trove/Gemfile index fc2214398..984392186 100644 --- a/trove/Gemfile +++ b/trove/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/trove/spec/spec_helper.rb b/trove/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/trove/spec/spec_helper.rb +++ b/trove/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' diff --git a/vswitch/Gemfile b/vswitch/Gemfile index fc2214398..984392186 100644 --- a/vswitch/Gemfile +++ b/vswitch/Gemfile @@ -13,7 +13,6 @@ group :development, :test do gem 'puppet-lint-variable_contains_upcase', :require => 'false' gem 'puppet-lint-numericvariable', :require => 'false' gem 'json', :require => 'false' - gem 'webmock', :require => 'false' end group :system_tests do diff --git a/vswitch/lib/puppet/provider/vs_port/ovs_redhat.rb b/vswitch/lib/puppet/provider/vs_port/ovs_redhat.rb index 0134bfa38..44da99929 100644 --- a/vswitch/lib/puppet/provider/vs_port/ovs_redhat.rb +++ b/vswitch/lib/puppet/provider/vs_port/ovs_redhat.rb @@ -8,10 +8,10 @@ desc 'Openvswitch port manipulation for RedHat OSes family' - self::BASE = '/etc/sysconfig/network-scripts/ifcfg-' + BASE ||= '/etc/sysconfig/network-scripts/ifcfg-' # When not seedling from interface file - self::DEFAULT = { + DEFAULT ||= { 'ONBOOT' => 'yes', 'BOOTPROTO' => 'dhcp', 'PEERDNS' => 'no', diff --git a/vswitch/spec/spec_helper.rb b/vswitch/spec/spec_helper.rb index 3df4cede1..172614e99 100644 --- a/vswitch/spec/spec_helper.rb +++ b/vswitch/spec/spec_helper.rb @@ -1,6 +1,5 @@ require 'puppetlabs_spec_helper/module_spec_helper' require 'shared_examples' -require 'webmock/rspec' RSpec.configure do |c| c.alias_it_should_behave_like_to :it_configures, 'configures' From 2ec36c34ba885250f87c51c16835abb7c35c872d Mon Sep 17 00:00:00 2001 From: Lukas Bezdicka Date: Mon, 1 Feb 2016 11:28:38 +0100 Subject: [PATCH 26/35] Drop puppet-galera Module puppet-galera is deprecated and no longer under development. Only composition layer that used it was patched [1]. [1] - https://review.openstack.org/#/c/273030/ Change-Id: I2bcc84906738bd99a07d66642816ec29a5503da8 (cherry picked from commit 2ca59abce1b1157420387ba552876a716121a16c) --- Puppetfile | 4 - galera/.fixtures.yml | 7 - galera/.gitignore | 6 - galera/Gemfile | 18 -- galera/LICENSE | 201 ------------------ galera/README.md | 43 ---- galera/Rakefile | 27 --- .../puppet/parser/functions/wsrep_options.rb | 9 - galera/manifests/monitor.pp | 90 -------- galera/manifests/server.pp | 155 -------------- galera/metadata.json | 51 ----- galera/spec/classes/galera_monitor_spec.rb | 81 ------- galera/spec/classes/galera_server_spec.rb | 60 ------ galera/spec/spec_helper.rb | 1 - galera/templates/clustercheck.erb | 7 - galera/templates/wsrep.cnf.erb | 133 ------------ 16 files changed, 893 deletions(-) delete mode 100644 galera/.fixtures.yml delete mode 100644 galera/.gitignore delete mode 100644 galera/Gemfile delete mode 100644 galera/LICENSE delete mode 100644 galera/README.md delete mode 100644 galera/Rakefile delete mode 100644 galera/lib/puppet/parser/functions/wsrep_options.rb delete mode 100644 galera/manifests/monitor.pp delete mode 100644 galera/manifests/server.pp delete mode 100644 galera/metadata.json delete mode 100644 galera/spec/classes/galera_monitor_spec.rb delete mode 100644 galera/spec/classes/galera_server_spec.rb delete mode 100644 galera/spec/spec_helper.rb delete mode 100644 galera/templates/clustercheck.erb delete mode 100644 galera/templates/wsrep.cnf.erb diff --git a/Puppetfile b/Puppetfile index 035e4e03f..513f30ac3 100644 --- a/Puppetfile +++ b/Puppetfile @@ -58,10 +58,6 @@ mod 'fluentd', :commit => 'ddc5f0e4c6c53d15f0cbd34f74bfaa91a0fb299f', :git => 'https://github.com/soylent/konstantin-fluentd.git' -mod 'galera', - :commit => 'a2ecf273aef9ef9193bba35c235fb7b8b821a0c3', - :git => 'https://github.com/redhat-openstack/puppet-galera.git' - mod 'git', :commit => '8e7f586', :git => 'https://github.com/puppetlabs/puppetlabs-git.git' diff --git a/galera/.fixtures.yml b/galera/.fixtures.yml deleted file mode 100644 index 989d203c2..000000000 --- a/galera/.fixtures.yml +++ /dev/null @@ -1,7 +0,0 @@ -fixtures: - repositories: - "stdlib": "git://github.com/puppetlabs/puppetlabs-stdlib" - "xinetd": "git://github.com/puppetlabs/puppetlabs-xinetd" - "mysql": "git://github.com/puppetlabs/puppetlabs-mysql" - symlinks: - "galera": "#{source_dir}" diff --git a/galera/.gitignore b/galera/.gitignore deleted file mode 100644 index bb757b24b..000000000 --- a/galera/.gitignore +++ /dev/null @@ -1,6 +0,0 @@ -Gemfile.lock -.vagrant -vendor/ -.bundle/ -spec/fixtures/modules -spec/fixtures/manifests/site.pp diff --git a/galera/Gemfile b/galera/Gemfile deleted file mode 100644 index 95232d551..000000000 --- a/galera/Gemfile +++ /dev/null @@ -1,18 +0,0 @@ -source 'https://rubygems.org' - -group :development, :test do - gem 'puppetlabs_spec_helper', :require => false - gem "rspec-puppet", :git => 'https://github.com/rodjek/rspec-puppet.git' - gem 'puppet-lint', :git => 'https://github.com/rodjek/puppet-lint.git' - gem 'puppet-lint-param-docs', '1.1.0' - gem 'puppet-syntax' - gem 'rake', :require => false -end - -if puppetversion = ENV['PUPPET_GEM_VERSION'] - gem 'puppet', puppetversion, :require => false -else - gem 'puppet', :require => false -end - -# vim:ft=ruby diff --git a/galera/LICENSE b/galera/LICENSE deleted file mode 100644 index 8d968b6cb..000000000 --- a/galera/LICENSE +++ /dev/null @@ -1,201 +0,0 @@ - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "[]" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright [yyyy] [name of copyright owner] - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. diff --git a/galera/README.md b/galera/README.md deleted file mode 100644 index 33da7f111..000000000 --- a/galera/README.md +++ /dev/null @@ -1,43 +0,0 @@ -# Galera module - -This is a module for installing and confiuring galera. - -It depends on the mysql module from puppetlabs as well as xinetd. - -## Usage - -### galera::server - - Used to configure a MariaDB Galera server cluster. - - class { 'galera::server': - wsrep_cluster_name => 'galera_cluster', - wsrep_sst_method => 'rsync' - wsrep_sst_username => 'ChangeMe', - wsrep_sst_password => 'ChangeMe', - } - -### galera::monitor - - Used to monitor a MariaDB Galera cluster server. The class is meant - to be used in a server load-balancer environment. - - class {'galera::monitor': - monitor_username => 'mon_user', - monitor_password => 'mon_pass' - } - - Here is a sample 3-node HAProxy Configuration: - - listen galera 192.168.220.40:3306 - balance leastconn - mode tcp - option tcpka - option httpchk - server control01 192.168.220.41:3306 check port 9200 inter 2000 rise 2 fall 5 - server control02 192.168.220.42:3306 check port 9200 inter 2000 rise 2 fall 5 - server control03 192.168.220.43:3306 check port 9200 inter 2000 rise 2 fall 5 - -## Authors - -Daneyon Hansen, Ryan O'Hara diff --git a/galera/Rakefile b/galera/Rakefile deleted file mode 100644 index c3bb5ac20..000000000 --- a/galera/Rakefile +++ /dev/null @@ -1,27 +0,0 @@ -require 'puppetlabs_spec_helper/rake_tasks' -require 'puppet-lint/tasks/puppet-lint' -require 'puppet-syntax/tasks/puppet-syntax' - -exclude_paths = [ - "pkg/**/*", - "vendor/**/*", - "spec/**/*", -] - -PuppetSyntax.exclude_paths = exclude_paths - -Rake::Task[:lint].clear -PuppetLint::RakeTask.new :lint do |config| - config.ignore_paths = exclude_paths - config.log_format = '%{path}:%{linenumber}:%{check}:%{KIND}:%{message}' - config.disable_checks = [ "class_inherits_from_params_class", "80chars" ] - config.fail_on_warnings = true - config.relative = true -end - -desc "Run syntax, lint, and spec tests." -task :test => [ - :syntax, - :lint, - :spec, -] diff --git a/galera/lib/puppet/parser/functions/wsrep_options.rb b/galera/lib/puppet/parser/functions/wsrep_options.rb deleted file mode 100644 index f09049026..000000000 --- a/galera/lib/puppet/parser/functions/wsrep_options.rb +++ /dev/null @@ -1,9 +0,0 @@ -module Puppet::Parser::Functions - newfunction(:wsrep_options, :type => :rvalue) do |args| - opts = args[0] - opts.delete_if {|key, val| val.equal? :undef} - opts.sort.map do |k,v| - String(k) + "=" + String(v) - end - end -end diff --git a/galera/manifests/monitor.pp b/galera/manifests/monitor.pp deleted file mode 100644 index cfef0c230..000000000 --- a/galera/manifests/monitor.pp +++ /dev/null @@ -1,90 +0,0 @@ -# == Class: galera::monitor -# -# === Parameters: -# -# [*mysql_username*] -# Username of the service account used for the clustercheck script. -# -# [*mysql_password*] -# Password of the service account used for the clustercheck script. -# -# [*mysql_host*] -# Hostname/IP address of mysql server to monitor. -# Defaults to 127.0.0.1. -# -# [*mysql_port*] -# Port used by mysql service. -# Defaults to 3306. -# -# [*monitor_port*] -# Port used by galera monitor service. -# Defaults to 9200. -# -# [*monitor_script*] -# Full path to monitor script.\ -# Defaults to '/usr/bin/clustercheck'. -# -# [*enabled*] -# Enable/Disable galera monitor xinetd::service. -# Defaults to true. -# -# [*create_mysql_user*] -# Whether to create mysql user or not. -# Defaults to false. -# -# === Actions: -# -# === Requires: -# -# === Sample usage: -# -# class { 'galera::monitor': -# mysql_username => 'mon_user', -# mysql_password => 'mon_pass' -# } -# -class galera::monitor ( - $mysql_username = 'monitor_user', - $mysql_password = 'monitor_pass', - $mysql_host = '127.0.0.1', - $mysql_port = '3306', - $monitor_port = '9200', - $monitor_script = '/usr/bin/clustercheck', - $enabled = true, - $create_mysql_user = false, -) { - - if $enabled { - $monitor_disable = 'no' - } else { - $monitor_disable = 'yes' - } - - file { '/etc/sysconfig/clustercheck': - mode => '0640', - content => template('galera/clustercheck.erb'), - owner => 'root', - group => 'root', - } - - xinetd::service { 'galera-monitor': - disable => $monitor_disable, - port => $monitor_port, - server => $monitor_script, - flags => 'REUSE', - per_source => 'UNLIMITED', - service_type => 'UNLISTED', - log_on_success => '', - log_on_success_operator => '=', - log_on_failure => 'HOST', - log_on_failure_operator => '=', - } - - if $create_mysql_user { - mysql_user { "${mysql_username}@${mysql_host}": - ensure => present, - password_hash => mysql_password($mysql_password), - require => [File['/root/.my.cnf'],Service['mysqld']], - } - } -} diff --git a/galera/manifests/server.pp b/galera/manifests/server.pp deleted file mode 100644 index eff45b14d..000000000 --- a/galera/manifests/server.pp +++ /dev/null @@ -1,155 +0,0 @@ -# == Class: galera::server -# -# manages the installation of the galera server. -# manages the package, service, galera.cnf -# -# === Parameters: -# -# [*bootstrap*] -# Defaults to false, boolean to set cluster boostrap. -# -# [*package_name*] -# The name of the galera package. -# -# [*package_ensure*] -# Ensure state for package. Can be specified as version. -# -# [*wsrep_node_address*] -# Address of local galera node. -# -# [*wsrep_provider*] -# Full path to wsrep provider library or 'none'. -# -# [*wsrep_cluster_name*] -# Logical cluster name. be the same for all nodes. -# -# [*wsrep_cluster_members*] -# List of cluster members, IP addresses or hostnames. -# -# [*wsrep_sst_method*] -# State snapshot transfer method. -# -# [*wsrep_sst_username*] -# Username used by the wsrep_sst_auth authentication string. -# -# [*wsrep_sst_password*] -# Password used by the wsrep_sst_auth authentication string. -# -# [*wsrep_ssl*] -# Boolean to disable SSL even if certificate and key are configured. -# -# [*wsrep_ssl_key*] -# Private key for the certificate above, unencrypted, in PEM format. -# -# [*wsrep_ssl_cert*] -# Certificate file in PEM format. -# -# [*debug*] -# -# [*wsrep_bind_address*] -# Address to bind galera service. -# Deprecated, please use ::mysql::server class. -# -# [*mysql_server_hash*] -# Hash of mysql server parameters. -# Deprecated, please use ::mysql::server class. -# -# -# [*manage_service*] -# State of the service. -# Deprecated, please use ::mysql::server class. -# -# [*service_name*] -# The name of the galera service. -# Deprecated, please use ::mysql::server class. -# -# [*service_enable*] -# Defaults to true, boolean to set service enable. -# Deprecated, please use ::mysql::server class. -# -# [*service_ensure*] -# Defaults to running, needed to set root password. -# Deprecated, please use ::mysql::server class. -# -# [*service_provider*] -# What service provider to use. -# Deprecated, please use ::mysql::server class. -# -# === Actions: -# -# === Requires: -# -# === Sample Usage: -# class { 'galera::server': -# wsrep_cluster_name => 'galera_cluster', -# wsrep_sst_method => 'rsync' -# wsrep_sst_username => 'ChangeMe', -# wsrep_sst_password => 'ChangeMe', -# } -# -class galera::server ( - $bootstrap = false, - $debug = false, - $wsrep_node_address = undef, - $wsrep_provider = '/usr/lib64/galera/libgalera_smm.so', - $wsrep_cluster_name = 'galera_cluster', - $wsrep_cluster_members = [ $::ipaddress ], - $wsrep_sst_method = 'rsync', - $wsrep_sst_username = 'root', - $wsrep_sst_password = undef, - $wsrep_ssl = false, - $wsrep_ssl_key = undef, - $wsrep_ssl_cert = undef, - $create_mysql_resource = true, - # DEPRECATED OPTIONS - $mysql_server_hash = {}, - $wsrep_bind_address = '0.0.0.0', - $manage_service = false, - $service_name = 'mariadb', - $service_enable = true, - $service_ensure = 'running', -) { - if $create_mysql_resource { - warning("DEPRECATED: ::mysql::server should be called manually, please set create_mysql_resource to false and call class ::mysql::server with your config") - - $mysql_server_class = { 'mysql::server' => $mysql_server_hash } - - create_resources( 'class', $mysql_server_class ) - } - - if $wsrep_bind_address { - warning("DEPRECATED: wsrep_bind_address is deprecated, you should use bind_address of mysql module") - } - - if $wsrep_ssl { - $wsrep_provider_options = wsrep_options({ - 'socket.ssl' => $wsrep_ssl, - 'socket.ssl_key' => $wsrep_ssl_key, - 'socket.ssl_cert' => $wsrep_ssl_cert, - }) - } else { - $wsrep_provider_options = wsrep_options({ - 'socket.ssl' => $wsrep_ssl, - }) - } - - $wsrep_debug = bool2num($debug) - - file { '/etc/my.cnf.d/galera.cnf': - ensure => present, - mode => '0644', - owner => 'root', - group => 'root', - content => template('galera/wsrep.cnf.erb'), - notify => Service['mysqld'], - } - - if $manage_service { - warning("DEPRECATED: service setup is deprecated, you should use mysql module for this.") - service { 'galera': - ensure => $service_ensure, - name => $service_name, - enable => $service_enable, - } - } -} diff --git a/galera/metadata.json b/galera/metadata.json deleted file mode 100644 index 26a8cb5af..000000000 --- a/galera/metadata.json +++ /dev/null @@ -1,51 +0,0 @@ -{ - "name": "puppet-galera", - "version": "0.0.2", - "author": "Ryan O'Hara", - "summary": "Install/Configure MariaDB with galera", - "license": "Apache 2.0", - "source": "git://github.com/redhat-openstack/puppet-galera.git", - "project_page": "https://github.com/redhat-openstack/puppet-galera", - "operatingsystem_support": [ - { - "operatingsystem": "RedHat", - "operatingsystemrelease": [ - "5", - "6", - "7" - ] - }, - { - "operatingsystem": "CentOS", - "operatingsystemrelease": [ - "5", - "6", - "7" - ] - }, - { - "operatingsystem": "Scientific", - "operatingsystemrelease": [ - "5", - "6", - "7" - ] - } - ], - "requirements": [ - { - "name": "pe", - "version_requirement": "3.x" - }, - { - "name": "puppet", - "version_requirement": "3.x" - } - ], - "description": "Install/Configure MariaDB with Galera", - "dependencies": [ - {"name":"puppetlabs/stdlib","version_requirement":">= 2.4.0"}, - {"name":"puppetlabs/mysql","version_requirement":">= 2.3.1"}, - {"name":"puppetlabs/xinetd","version_requirement":">= 1.2.0"} - ] -} diff --git a/galera/spec/classes/galera_monitor_spec.rb b/galera/spec/classes/galera_monitor_spec.rb deleted file mode 100644 index 3bdf322fb..000000000 --- a/galera/spec/classes/galera_monitor_spec.rb +++ /dev/null @@ -1,81 +0,0 @@ -require 'spec_helper' -describe 'galera::monitor' do - - let :facts do - { :osfamily => 'RedHat' } - end - - let :params do - { - :mysql_username => 'monitor_user', - :mysql_password => 'monitor_pass', - :mysql_host => '127.0.0.1', - :mysql_port => '3306', - :monitor_port => '9200', - :monitor_script => '/usr/bin/clustercheck', - :enabled => true, - :create_mysql_user => false, - } - end - - context 'with galera-monitor xinetd service' do - context 'with enabled parameter to true' do - it 'Configures galera-monitor xinetd service' do - should contain_xinetd__service('galera-monitor').with( - :disable => 'no', - :port => '9200', - :server => '/usr/bin/clustercheck', - :flags => 'REUSE', - :per_source => 'UNLIMITED', - :service_type => 'UNLISTED', - :log_on_success => '', - :log_on_success_operator => '=', - :log_on_failure => 'HOST', - :log_on_failure_operator => '=', - ) - end - end - - context 'with enabled parameter to false' do - let(:params) { {:enabled => false } } - it 'Configures galera-monitor xinetd service' do - should contain_xinetd__service('galera-monitor').with( - :disable => 'yes', - :port => '9200', - :server => '/usr/bin/clustercheck', - :flags => 'REUSE', - :per_source => 'UNLIMITED', - :service_type => 'UNLISTED', - :log_on_success => '', - :log_on_success_operator => '=', - :log_on_failure => 'HOST', - :log_on_failure_operator => '=', - ) - end - end - end - - it 'Configures clustercheck configuration file' do - should contain_file('/etc/sysconfig/clustercheck').with( - :mode => '0640', - :owner => 'root', - :group => 'root', - ) - end - - context 'with create_mysql_user parameter' do - let(:params) { {:create_mysql_user => false} } - context 'create_mysql_user to false' do - it 'Should not contain mysql_user resource' do - should_not contain_mysql_user() - end - end - - let(:params) { {:create_mysql_user => true} } - context 'create_mysql_user to true' do - it 'Should contain mysql_user resource' do - should contain_mysql_user("monitor_user@127.0.0.1") - end - end - end -end diff --git a/galera/spec/classes/galera_server_spec.rb b/galera/spec/classes/galera_server_spec.rb deleted file mode 100644 index 41654f19a..000000000 --- a/galera/spec/classes/galera_server_spec.rb +++ /dev/null @@ -1,60 +0,0 @@ -require 'spec_helper' - -describe 'galera::server', :type => :class do - - let :facts do - { :osfamily => 'RedHat' } - end - - let :params do - { - :bootstrap => false, - :debug => false, - :service_name => 'mariadb', - :service_enable => true, - :service_ensure => 'running', - :manage_service => false, - :wsrep_bind_address => '0.0.0.0', - :wsrep_node_address => 'undef', - :wsrep_provider => '/usr/lib64/galera/libgalera_smm.so', - :wsrep_cluster_name => 'galera_cluster', - :wsrep_cluster_members => ['127.0.0.1'], - :wsrep_sst_method => 'rsync', - :wsrep_sst_username => 'root', - :wsrep_sst_password => 'undef', - :wsrep_ssl => false, - :wsrep_ssl_key => 'undef', - :wsrep_ssl_cert => 'undef', - } - end - - it { should contain_class('mysql::server') } - - context 'Configures /etc/my.cnf.d/galera.cnf' do - it { should contain_file('/etc/my.cnf.d/galera.cnf').with( - 'ensure' => 'present', - 'mode' => '0644', - 'owner' => 'root', - 'group' => 'root', - 'notify' => 'Service[mariadb]' - ) - } - end - - context 'with manage_service to false' do - it "Doesn't configure galera service" do - should_not contain_service('galera') - end - end - - context 'with manage_service to true' do - let(:params) { {:manage_service => true} } - it "Configures galera service" do - should contain_service('galera').with( - 'ensure' => 'running', - 'name' => 'mariadb', - 'enable' => 'true', - ) - end - end -end diff --git a/galera/spec/spec_helper.rb b/galera/spec/spec_helper.rb deleted file mode 100644 index 2c6f56649..000000000 --- a/galera/spec/spec_helper.rb +++ /dev/null @@ -1 +0,0 @@ -require 'puppetlabs_spec_helper/module_spec_helper' diff --git a/galera/templates/clustercheck.erb b/galera/templates/clustercheck.erb deleted file mode 100644 index 3abac52c5..000000000 --- a/galera/templates/clustercheck.erb +++ /dev/null @@ -1,7 +0,0 @@ -# This file is being maintained by Puppet. -# DO NOT EDIT - -MYSQL_USERNAME="<%= @mysql_username %>" -MYSQL_PASSWORD="<%= @mysql_password %>" -MYSQL_HOST="<%= @mysql_host %>" -MYSQL_PORT="<%= @mysql_port %>" diff --git a/galera/templates/wsrep.cnf.erb b/galera/templates/wsrep.cnf.erb deleted file mode 100644 index b471f627d..000000000 --- a/galera/templates/wsrep.cnf.erb +++ /dev/null @@ -1,133 +0,0 @@ -# This file contains wsrep-related mysqld options. It should be included -# in the main MySQL configuration file. -# -# Options that need to be customized: -# - wsrep_provider -# - wsrep_cluster_address -# - wsrep_sst_auth -# The rest of defaults should work out of the box. - -## -## mysqld options _MANDATORY_ for correct opration of the cluster -## -[mysqld] - -# (This must be substituted by wsrep_format) -binlog_format=ROW - -# Currently only InnoDB storage engine is supported -default-storage-engine=innodb - -# to avoid issues with 'bulk mode inserts' using autoinc -innodb_autoinc_lock_mode=2 - -# This is a must for paralell applying -innodb_locks_unsafe_for_binlog=1 - -# Query Cache is not supported with wsrep -#### query_cache_size=0 now in server.cnf -query_cache_type=0 - -# Override bind-address -# In some systems bind-address defaults to 127.0.0.1, and with mysqldump SST -# it will have (most likely) disastrous consequences on donor node -bind-address=<%= @wsrep_bind_address %> - -## -## WSREP options -## - -# Full path to wsrep provider library or 'none' -wsrep_provider=<%= @wsrep_provider %> - -# Provider specific configuration options -wsrep_provider_options="<%= @wsrep_provider_options.join '; ' %>" - -# Logical cluster name. Should be the same for all nodes. -wsrep_cluster_name="<%= @wsrep_cluster_name %>" - -# Human-readable node name (non-unique). Hostname by default. -#wsrep_node_name= - -# Base replication [:port] of the node. -# The values supplied will be used as defaults for state transfer receiving, -# listening ports and so on. Default: address of the first network interface. -<% if @wsrep_node_address -%> -wsrep_node_address=<%= @wsrep_node_address %> -<% else -%> -#wsrep_node_address= -<% end -%> - -# Address for incoming client connections. Autodetect by default. -#wsrep_node_incoming_address= - -# How many threads will process writesets from other nodes -wsrep_slave_threads=1 - -# DBUG options for wsrep provider -#wsrep_dbug_option - -# Generate fake primary keys for non-PK tables (required for multi-master -# and parallel applying operation) -wsrep_certify_nonPK=1 - -# Maximum number of rows in write set -wsrep_max_ws_rows=131072 - -# Maximum size of write set -wsrep_max_ws_size=1073741824 - -# to enable debug level logging, set this to 1 -wsrep_debug=<%= @wsrep_debug %> - -# convert locking sessions into transactions -wsrep_convert_LOCK_to_trx=0 - -# how many times to retry deadlocked autocommits -wsrep_retry_autocommit=1 - -# change auto_increment_increment and auto_increment_offset automatically -wsrep_auto_increment_control=1 - -# retry autoinc insert, which failed for duplicate key error -wsrep_drupal_282555_workaround=0 - -# enable "strictly synchronous" semantics for read operations -wsrep_causal_reads=0 - -# Command to call when node status or cluster membership changes. -# Will be passed all or some of the following options: -# --status - new status of this node -# --uuid - UUID of the cluster -# --primary - whether the component is primary or not ("yes"/"no") -# --members - comma-separated list of members -# --index - index of this node in the list -wsrep_notify_cmd= - -## -## WSREP State Transfer options -## - -# State Snapshot Transfer method -wsrep_sst_method=<%= @wsrep_sst_method %> - -# Address which donor should send State Snapshot to. -# Should be the address of THIS node. DON'T SET IT TO DONOR ADDRESS!!! -# (SST method dependent. Defaults to the first IP of the first interface) -#wsrep_sst_receive_address= - -# SST authentication string. This will be used to send SST to joining nodes. -# Depends on SST method. For mysqldump method it is root: -# Below is ignored if using rsync for wsrep_sst_method -wsrep_sst_auth=<%= @wsrep_sst_username %>:<%= @wsrep_sst_password %> - -# Desired SST donor name. -#wsrep_sst_donor= - -# Reject client queries when donating SST (false) -#wsrep_sst_donor_rejects_queries=0 - -# Protocol version to use -# wsrep_protocol_version= - -skip-name-resolve=1 From cfc7ac6136a7f0dfeb59d7ab9aa60d44bafb4cee Mon Sep 17 00:00:00 2001 From: Gael Chamoulaud Date: Tue, 16 Feb 2016 14:28:00 +0100 Subject: [PATCH 27/35] Update puppet-pacemaker url puppet-pacemaker is now part of OpenStack and is hosted on openstack/puppet-pacemaker namespace, instead of redhat-openstack/puppet-pacemaker. Change-Id: I28a3add8c1ccc7a3e847ea34b3bd8642fd405d54 Fixes: rhbz#1308710 Signed-off-by: Gael Chamoulaud --- Puppetfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Puppetfile b/Puppetfile index 513f30ac3..809a18638 100644 --- a/Puppetfile +++ b/Puppetfile @@ -184,7 +184,7 @@ mod 'openstacklib', mod 'pacemaker', :commit => '00d99b66931d520af202b94d0a2fe3b43f11d8ee', - :git => 'https://github.com/redhat-openstack/puppet-pacemaker.git' + :git => 'https://github.com/openstack/puppet-pacemaker.git' mod 'puppet', :commit => 'ede26b4d712697ea3edc91a40a0e8361b2ad0374', From bb5fae060774e50dec271a7028e48f172c612683 Mon Sep 17 00:00:00 2001 From: Jason Guiditta Date: Tue, 16 Feb 2016 10:07:53 -0500 Subject: [PATCH 28/35] Sync of liberty puppet modules. Update gnocchi to b8b942990fef44045e2e678ba3f7901d62f1fdb8 b8b942990fef44045e2e678ba3f7901d62f1fdb8 Add missing metricd service Change-Id: I521388e2c89f7c5bde40639d55aafd3f8e244e32 Update ironic to 754a156d6a9f1175a454789c53f2e288e7ec8e21 754a156d6a9f1175a454789c53f2e288e7ec8e21 [inspector] allow changing list of processing hooks Change-Id: I63be90ea2f595b2e1c8330f883850726a6968003 Update neutron to a83bfe12562273a9e7bbba3ade9d8028bb6edd9d a83bfe12562273a9e7bbba3ade9d8028bb6edd9d Merge "Change of PLUMgrid plugin package and addition of new parameters." into stable/liberty bb3a2c8fd41dfca79b019d448c58e518387ab546 Metadata agent config should be applied after package installation ea045e366490c1a2a651fd237f5cd94269ed0205 Change of PLUMgrid plugin package and addition of new parameters. Change-Id: Ia894cca596697c5a3a9711d68d3e156c647a2051 Update tripleo to bd23dab3bbc710c72c481953f2cf13f8c51120ad bd23dab3bbc710c72c481953f2cf13f8c51120ad Merge "SSL/Cinder: enable ssl_header_handler filter" into stable/liberty 9f3a822d2bd1e70fbec5ba9aa97c71860fc6bb49 Merge "Remove all 'validate_array' statements" into stable/liberty 2fda01c6977cf9cc81fc3d09c6e8f5193a23028b Merge "Fix unit tests failing against Puppet 4.3.x" into stable/liberty 225117948b1d528ad03002a4aba6228c0c3ff1f6 Fix unit tests failing against Puppet 4.3.x 540d45b3cb6925a5ccf1da552f1dfd4c0f657f3d SSL/Cinder: enable ssl_header_handler filter 44cd7265a82d8a9cdff16fa57061523cbf1d33e9 Remove all 'validate_array' statements Change-Id: Ib9e4621d258b4ac73f516215fa2a67a1972dc9b1 Update openstacklib to 3c03db279240c0f871a579331f41c3a3e0fc6b52 3c03db279240c0f871a579331f41c3a3e0fc6b52 Utility to handle IPv6 address brackets. Change-Id: I5eaf7c16b14493a5cf5ee252848dc91397fff34a --- Puppetfile | 10 +-- gnocchi/manifests/metricd.pp | 49 +++++++++++ gnocchi/manifests/params.pp | 4 + gnocchi/spec/acceptance/basic_gnocchi_spec.rb | 1 + gnocchi/spec/classes/gnocchi_metricd_spec.rb | 87 +++++++++++++++++++ ironic/manifests/inspector.pp | 8 ++ ironic/spec/classes/ironic_inspector_spec.rb | 21 +++-- neutron/manifests/agents/metadata.pp | 1 + neutron/manifests/params.pp | 4 +- neutron/manifests/plugins/plumgrid.pp | 47 +++++++++- .../classes/neutron_plugins_plumgrid_spec.rb | 9 +- .../parser/functions/normalize_ip_for_uri.rb | 31 +++++++ .../functions/normalize_ip_for_uri_spec.rb | 13 +++ tripleo/manifests/cluster/cassandra.pp | 7 +- tripleo/manifests/cluster/zookeeper.pp | 8 +- tripleo/manifests/network/midonet/agent.pp | 9 +- tripleo/manifests/network/midonet/api.pp | 7 +- tripleo/manifests/ssl/cinder_config.pp | 28 ++++++ .../classes/tripleo_midonet_agent_spec.rb | 1 + 19 files changed, 318 insertions(+), 27 deletions(-) create mode 100644 gnocchi/manifests/metricd.pp create mode 100644 gnocchi/spec/classes/gnocchi_metricd_spec.rb create mode 100644 openstacklib/lib/puppet/parser/functions/normalize_ip_for_uri.rb create mode 100644 openstacklib/spec/functions/normalize_ip_for_uri_spec.rb create mode 100644 tripleo/manifests/ssl/cinder_config.pp diff --git a/Puppetfile b/Puppetfile index 809a18638..8d943af4d 100644 --- a/Puppetfile +++ b/Puppetfile @@ -71,7 +71,7 @@ mod 'gluster', :git => 'https://github.com/purpleidea/puppet-gluster.git' mod 'gnocchi', - :commit => 'e774f7df73ccce96ff2fe256e40a9de078b0fffb', + :commit => 'b8b942990fef44045e2e678ba3f7901d62f1fdb8', :git => 'https://github.com/openstack/puppet-gnocchi.git' mod 'haproxy', @@ -91,7 +91,7 @@ mod 'inifile', :git => 'https://github.com/puppetlabs/puppetlabs-inifile.git' mod 'ironic', - :commit => '92ba5d6f7be0f1a24664888f4a8767ab3ab9e703', + :commit => '754a156d6a9f1175a454789c53f2e288e7ec8e21', :git => 'https://github.com/openstack/puppet-ironic.git' mod 'java', @@ -155,7 +155,7 @@ mod 'nagios', :git => 'https://github.com/gildub/puppet-nagios-openstack.git' mod 'neutron', - :commit => '3794e88c4e6cbf3a14125f19a1be9fa862718583', + :commit => 'a83bfe12562273a9e7bbba3ade9d8028bb6edd9d', :git => 'https://github.com/openstack/puppet-neutron.git' mod 'nova', @@ -179,7 +179,7 @@ mod 'openstack_extras', :git => 'https://github.com/openstack/puppet-openstack_extras.git' mod 'openstacklib', - :commit => 'd36b88673a893284d5898159cc39ab93bc18c00c', + :commit => '3c03db279240c0f871a579331f41c3a3e0fc6b52', :git => 'https://github.com/openstack/puppet-openstacklib.git' mod 'pacemaker', @@ -255,7 +255,7 @@ mod 'tomcat', :git => 'https://github.com/puppetlabs/puppetlabs-tomcat.git' mod 'tripleo', - :commit => '78e62660da958a975a899c5866a89d8158020222', + :commit => 'bd23dab3bbc710c72c481953f2cf13f8c51120ad', :git => 'https://github.com/openstack/puppet-tripleo.git' mod 'trove', diff --git a/gnocchi/manifests/metricd.pp b/gnocchi/manifests/metricd.pp new file mode 100644 index 000000000..76b2c7126 --- /dev/null +++ b/gnocchi/manifests/metricd.pp @@ -0,0 +1,49 @@ +# Installs & configure the gnocchi metricd service +# +# == Parameters +# +# [*enabled*] +# (optional) Should the service be enabled. +# Defaults to true +# +# [*package_ensure*] +# (optional) ensure state for package. +# Defaults to 'present' +# +# [*manage_service*] +# (optional) Whether the service should be managed by Puppet. +# Defaults to true. +# +class gnocchi::metricd ( + $manage_service = true, + $enabled = true, + $package_ensure = 'present', +) inherits gnocchi::params { + + Gnocchi_config<||> ~> Service['gnocchi-metricd'] + Package['gnocchi-metricd'] -> Service['gnocchi-metricd'] + + package { 'gnocchi-metricd': + ensure => $package_ensure, + name => $::gnocchi::params::metricd_package_name, + tag => ['openstack', 'gnocchi-package'], + } + + if $manage_service { + if $enabled { + $service_ensure = 'running' + } else { + $service_ensure = 'stopped' + } + } + + service { 'gnocchi-metricd': + ensure => $service_ensure, + name => $::gnocchi::params::metricd_service_name, + enable => $enabled, + hasstatus => true, + hasrestart => true, + tag => ['gnocchi-service', 'gnocchi-db-sync-service'], + } + +} diff --git a/gnocchi/manifests/params.pp b/gnocchi/manifests/params.pp index 0308dd3e2..4e2f43532 100644 --- a/gnocchi/manifests/params.pp +++ b/gnocchi/manifests/params.pp @@ -10,6 +10,8 @@ $api_service_name = 'openstack-gnocchi-api' $indexer_package_name = 'openstack-gnocchi-indexer-sqlalchemy' $carbonara_package_name = 'openstack-gnocchi-carbonara' + $metricd_package_name = 'openstack-gnocchi-metricd' + $metricd_service_name = 'openstack-gnocchi-metricd' $statsd_package_name = 'openstack-gnocchi-statsd' $statsd_service_name = 'openstack-gnocchi-statsd' $client_package_name = 'python-gnocchiclient' @@ -23,6 +25,8 @@ $api_service_name = 'gnocchi-api' $indexer_package_name = 'gnocchi-indexer-sqlalchemy' $carbonara_package_name = 'gnocchi-carbonara' + $metricd_package_name = 'gnocchi-metricd' + $metricd_service_name = 'gnocchi-metricd' $statsd_package_name = 'gnocchi-statsd' $statsd_service_name = 'gnocchi-statsd' $client_package_name = 'python-gnocchiclient' diff --git a/gnocchi/spec/acceptance/basic_gnocchi_spec.rb b/gnocchi/spec/acceptance/basic_gnocchi_spec.rb index b93abe320..5c284a498 100644 --- a/gnocchi/spec/acceptance/basic_gnocchi_spec.rb +++ b/gnocchi/spec/acceptance/basic_gnocchi_spec.rb @@ -33,6 +33,7 @@ class { '::gnocchi::api': keystone_identity_uri => 'http://127.0.0.1:35357/', service_name => 'httpd', } + class { '::gnocchi::metricd': } class { '::gnocchi::db::sync': } class { '::gnocchi::storage': } class { '::gnocchi::storage::file': } diff --git a/gnocchi/spec/classes/gnocchi_metricd_spec.rb b/gnocchi/spec/classes/gnocchi_metricd_spec.rb new file mode 100644 index 000000000..10ee36f80 --- /dev/null +++ b/gnocchi/spec/classes/gnocchi_metricd_spec.rb @@ -0,0 +1,87 @@ +require 'spec_helper' + +describe 'gnocchi::metricd' do + + let :params do { } + end + + shared_examples_for 'gnocchi-metricd' do + + it { is_expected.to contain_class('gnocchi::params') } + + it 'installs gnocchi-metricd package' do + is_expected.to contain_package('gnocchi-metricd').with( + :ensure => 'present', + :name => platform_params[:metricd_package_name], + :tag => ['openstack', 'gnocchi-package'], + ) + end + + [{:enabled => true}, {:enabled => false}].each do |param_hash| + context "when service should be #{param_hash[:enabled] ? 'enabled' : 'disabled'}" do + before do + params.merge!(param_hash) + end + + it 'configures gnocchi-metricd service' do + is_expected.to contain_service('gnocchi-metricd').with( + :ensure => params[:enabled] ? 'running' : 'stopped', + :name => platform_params[:metricd_service_name], + :enable => params[:enabled], + :hasstatus => true, + :hasrestart => true, + :tag => ['gnocchi-service', 'gnocchi-db-sync-service'], + ) + end + end + end + + context 'with disabled service managing' do + before do + params.merge!({ + :manage_service => false, + :enabled => false }) + end + + it 'configures gnocchi-metricd service' do + is_expected.to contain_service('gnocchi-metricd').with( + :ensure => nil, + :name => platform_params[:metricd_service_name], + :enable => false, + :hasstatus => true, + :hasrestart => true, + :tag => ['gnocchi-service', 'gnocchi-db-sync-service'], + ) + end + end + end + + context 'on Debian platforms' do + let :facts do + { :osfamily => 'Debian', + :operatingsystem => 'Debian'} + end + + let :platform_params do + { :metricd_package_name => 'gnocchi-metricd', + :metricd_service_name => 'gnocchi-metricd' } + end + + it_configures 'gnocchi-metricd' + end + + context 'on RedHat platforms' do + let :facts do + { :osfamily => 'RedHat', + :operatingsystem => 'RedHat'} + end + + let :platform_params do + { :metricd_package_name => 'openstack-gnocchi-metricd', + :metricd_service_name => 'openstack-gnocchi-metricd' } + end + + it_configures 'gnocchi-metricd' + end + +end diff --git a/ironic/manifests/inspector.pp b/ironic/manifests/inspector.pp index dc7829e3b..65b6756dc 100644 --- a/ironic/manifests/inspector.pp +++ b/ironic/manifests/inspector.pp @@ -134,6 +134,10 @@ # Comma-separated list of IPA inspection collectors # Defaults to 'default' # +# [*additional_processing_hooks*] +# Comma-separated list of processing hooks to append to the default list. +# Defaults to undef +# class ironic::inspector ( $package_ensure = 'present', $enabled = true, @@ -164,6 +168,7 @@ $dnsmasq_local_ip = '192.168.0.1', $sync_db = true, $ramdisk_collectors = 'default', + $additional_processing_hooks = undef, ) { include ::ironic::params @@ -235,6 +240,9 @@ 'swift/password': value => $swift_password, secret => true; 'swift/tenant_name': value => $swift_tenant_name; 'swift/os_auth_url': value => $swift_auth_url; + # Here we use oslo.config interpolation with another option default_processing_hooks, + # which we don't change as it might break introspection completely. + 'processing/processing_hooks': value => join(delete_undef_values(['$default_processing_hooks', $additional_processing_hooks]), ','); } # Install package diff --git a/ironic/spec/classes/ironic_inspector_spec.rb b/ironic/spec/classes/ironic_inspector_spec.rb index 9013a2d9f..ae9dbe8a2 100644 --- a/ironic/spec/classes/ironic_inspector_spec.rb +++ b/ironic/spec/classes/ironic_inspector_spec.rb @@ -103,6 +103,7 @@ is_expected.to contain_ironic_inspector_config('swift/username').with_value(p[:swift_username]) is_expected.to contain_ironic_inspector_config('swift/tenant_name').with_value(p[:swift_tenant_name]) is_expected.to contain_ironic_inspector_config('swift/os_auth_url').with_value(p[:swift_auth_url]) + is_expected.to contain_ironic_inspector_config('processing/processing_hooks').with_value('$default_processing_hooks') end it 'should contain file /etc/ironic-inspector/dnsmasq.conf' do @@ -132,15 +133,16 @@ context 'when overriding parameters' do before :each do params.merge!( - :debug => true, - :auth_uri => 'http://192.168.0.1:5000/v2.0', - :identity_uri => 'http://192.168.0.1:35357', - :admin_password => 'password', - :ironic_password => 'password', - :ironic_auth_url => 'http://192.168.0.1:5000/v2.0', - :swift_password => 'password', - :swift_auth_url => 'http://192.168.0.1:5000/v2.0', - :pxe_transfer_protocol => 'http', + :debug => true, + :auth_uri => 'http://192.168.0.1:5000/v2.0', + :identity_uri => 'http://192.168.0.1:35357', + :admin_password => 'password', + :ironic_password => 'password', + :ironic_auth_url => 'http://192.168.0.1:5000/v2.0', + :swift_password => 'password', + :swift_auth_url => 'http://192.168.0.1:5000/v2.0', + :pxe_transfer_protocol => 'http', + :additional_processing_hooks => 'hook1,hook2', ) end it 'should replace default parameter with new value' do @@ -152,6 +154,7 @@ is_expected.to contain_ironic_inspector_config('ironic/os_auth_url').with_value(p[:ironic_auth_url]) is_expected.to contain_ironic_inspector_config('swift/password').with_value(p[:swift_password]) is_expected.to contain_ironic_inspector_config('swift/os_auth_url').with_value(p[:swift_auth_url]) + is_expected.to contain_ironic_inspector_config('processing/processing_hooks').with_value('$default_processing_hooks,hook1,hook2') end it 'should contain file /etc/ironic-inspector/dnsmasq.conf' do diff --git a/neutron/manifests/agents/metadata.pp b/neutron/manifests/agents/metadata.pp index f4a7f13d3..c674a1309 100644 --- a/neutron/manifests/agents/metadata.pp +++ b/neutron/manifests/agents/metadata.pp @@ -144,6 +144,7 @@ if $::neutron::params::metadata_agent_package { Package['neutron-metadata'] -> Service['neutron-metadata'] + Package['neutron-metadata'] -> Neutron_metadata_agent_config<||> package { 'neutron-metadata': ensure => $package_ensure, name => $::neutron::params::metadata_agent_package, diff --git a/neutron/manifests/params.pp b/neutron/manifests/params.pp index debecdcbf..906828c58 100644 --- a/neutron/manifests/params.pp +++ b/neutron/manifests/params.pp @@ -42,7 +42,7 @@ $midonet_server_package = 'python-neutron-plugin-midonet' $midonet_config_file = '/etc/neutron/plugins/midonet/midonet.ini' - $plumgrid_plugin_package = 'openstack-neutron-plumgrid' + $plumgrid_plugin_package = 'networking-plumgrid' $plumgrid_pythonlib_package = 'plumgrid-pythonlib' $plumgrid_config_file = '/etc/neutron/plugins/plumgrid/plumgrid.ini' @@ -125,7 +125,7 @@ $midonet_server_package = 'python-neutron-plugin-midonet' $midonet_config_file = '/etc/neutron/plugins/midonet/midonet.ini' - $plumgrid_plugin_package = 'neutron-plugin-plumgrid' + $plumgrid_plugin_package = 'networking-plumgrid' $plumgrid_pythonlib_package = 'plumgrid-pythonlib' $plumgrid_config_file = '/etc/neutron/plugins/plumgrid/plumgrid.ini' diff --git a/neutron/manifests/plugins/plumgrid.pp b/neutron/manifests/plugins/plumgrid.pp index 07376ce8c..13f81373b 100644 --- a/neutron/manifests/plugins/plumgrid.pp +++ b/neutron/manifests/plugins/plumgrid.pp @@ -12,9 +12,11 @@ # # [*username*] # PLUMgrid platform username +# Defaults to undef # # [*password*] # PLUMgrid platform password +# Defaults to undef # # [*servertimeout*] # Request timeout duration (seconds) to PLUMgrid paltform @@ -26,6 +28,7 @@ # # [*admin_password*] # Keystone admin password +# Defaults to undef # # [*controller_priv_host*] # Controller private host IP @@ -35,6 +38,10 @@ # Authorization protocol # Defaults to http # +# [*identity_version*] +# Keystone identity version +# Defaults to v2.0 +# # [*nova_metadata_ip*] # Nova metadata IP # Defaults to 127.0.0.1 @@ -45,9 +52,30 @@ # # [*metadata_proxy_shared_secret*] # Neutron metadata shared secret key +# Defaults to undef +# +# [*connector_type*] +# Neutron network connector type +# Defaults to distributed +# +# [*l2gateway_vendor*] +# L2 gateway vendor +# Defaults to undef +# +# [*l2gateway_sw_username*] +# L2 gateway username +# Defaults to undef +# +# [*l2gateway_sw_password*] +# L2 gateway password +# Defaults to undef +# +# [*plumlib_package_ensure*] +# (optional) Ensure state for plumlib package. +# Defaults to 'present'. # # [*package_ensure*] -# (optional) Ensure state for package. +# (optional) Ensure state for plugin package. # Defaults to 'present'. # class neutron::plugins::plumgrid ( @@ -60,9 +88,15 @@ $admin_password = undef, $controller_priv_host = '127.0.0.1', $auth_protocol = 'http', + $identity_version = 'v2.0', $nova_metadata_ip = '127.0.0.1', $nova_metadata_port = '8775', $metadata_proxy_shared_secret = undef, + $connector_type = 'distributed', + $l2gateway_vendor = undef, + $l2gateway_sw_username = undef, + $l2gateway_sw_password = undef, + $plumlib_package_ensure = 'present', $package_ensure = 'present' ) { @@ -70,6 +104,7 @@ Neutron_plugin_plumgrid<||> ~> Service['neutron-server'] Neutron_plumlib_plumgrid<||> ~> Service['neutron-server'] + Neutron_plugin_plumgrid<||> ~> Exec<| title == 'neutron-db-sync' |> ensure_resource('file', '/etc/neutron/plugins/plumgrid', { ensure => directory, @@ -94,10 +129,9 @@ } package { 'neutron-plumlib-plumgrid': - ensure => $package_ensure, + ensure => $plumlib_package_ensure, name => $::neutron::params::plumgrid_pythonlib_package } - warning('neutron-plumlib-plumgrid package management is deprecated, it will be dropped in a future release.') if $::osfamily == 'Debian' { file_line { '/etc/default/neutron-server:NEUTRON_PLUGIN_CONFIG': @@ -129,12 +163,17 @@ neutron_plumlib_plumgrid { 'keystone_authtoken/admin_user' : value => 'admin'; 'keystone_authtoken/admin_password': value => $admin_password, secret =>true; - 'keystone_authtoken/auth_uri': value => "${auth_protocol}://${controller_priv_host}:35357/v2.0"; + 'keystone_authtoken/auth_uri': value => "${auth_protocol}://${controller_priv_host}:35357/${identity_version}"; 'keystone_authtoken/admin_tenant_name': value => 'admin'; + 'keystone_authtoken/identity_version': value => $identity_version; 'PLUMgridMetadata/enable_pg_metadata' : value => 'True'; 'PLUMgridMetadata/metadata_mode': value => 'local'; 'PLUMgridMetadata/nova_metadata_ip': value => $nova_metadata_ip; 'PLUMgridMetadata/nova_metadata_port': value => $nova_metadata_port; 'PLUMgridMetadata/metadata_proxy_shared_secret': value => $metadata_proxy_shared_secret; + 'ConnectorType/connector_type': value => $connector_type; + 'l2gateway/vendor': value => $l2gateway_vendor; + 'l2gateway/sw_username': value => $l2gateway_sw_username; + 'l2gateway/sw_password': value => $l2gateway_sw_password; } } diff --git a/neutron/spec/classes/neutron_plugins_plumgrid_spec.rb b/neutron/spec/classes/neutron_plugins_plumgrid_spec.rb index 0788f0a37..b76f27e32 100644 --- a/neutron/spec/classes/neutron_plugins_plumgrid_spec.rb +++ b/neutron/spec/classes/neutron_plugins_plumgrid_spec.rb @@ -15,8 +15,10 @@ class { 'neutron': rabbit_password => 'passw0rd' }" :connection => 'http://127.0.0.1:35357/v2.0', :controller_priv_host => '127.0.0.1', :auth_protocol => 'http', + :identity_version => 'v2.0', :nova_metadata_ip => '127.0.0.1', :nova_metadata_port => '8775', + :connector_type => 'distributed', } end @@ -61,13 +63,18 @@ class { 'neutron': rabbit_password => 'passw0rd' }" is_expected.to contain_neutron_plumlib_plumgrid('keystone_authtoken/admin_user').with_value('admin') is_expected.to contain_neutron_plumlib_plumgrid('keystone_authtoken/admin_password').with_value(params[:admin_password]) is_expected.to contain_neutron_plumlib_plumgrid('keystone_authtoken/admin_tenant_name').with_value('admin') - auth_uri = params[:auth_protocol] + "://" + params[:controller_priv_host] + ":" + "35357/v2.0"; + auth_uri = params[:auth_protocol] + "://" + params[:controller_priv_host] + ":" + "35357/" + params[:identity_version]; is_expected.to contain_neutron_plumlib_plumgrid('keystone_authtoken/auth_uri').with_value(auth_uri) + is_expected.to contain_neutron_plumlib_plumgrid('keystone_authtoken/identity_version').with_value(params[:identity_version]) is_expected.to contain_neutron_plumlib_plumgrid('PLUMgridMetadata/enable_pg_metadata').with_value('True') is_expected.to contain_neutron_plumlib_plumgrid('PLUMgridMetadata/metadata_mode').with_value('local') is_expected.to contain_neutron_plumlib_plumgrid('PLUMgridMetadata/nova_metadata_ip').with_value(params[:nova_metadata_ip]) is_expected.to contain_neutron_plumlib_plumgrid('PLUMgridMetadata/nova_metadata_port').with_value(params[:nova_metadata_port]) is_expected.to contain_neutron_plumlib_plumgrid('PLUMgridMetadata/metadata_proxy_shared_secret').with_value(params[:metadata_proxy_shared_secret]) + is_expected.to contain_neutron_plumlib_plumgrid('ConnectorType/connector_type').with_value('distributed') + is_expected.to contain_neutron_plumlib_plumgrid('l2gateway/vendor').with_value(params[:l2gateway_vendor]) + is_expected.to contain_neutron_plumlib_plumgrid('l2gateway/sw_username').with_value(params[:l2gateway_sw_username]) + is_expected.to contain_neutron_plumlib_plumgrid('l2gateway/sw_password').with_value(params[:l2gateway_sw_password]) end end diff --git a/openstacklib/lib/puppet/parser/functions/normalize_ip_for_uri.rb b/openstacklib/lib/puppet/parser/functions/normalize_ip_for_uri.rb new file mode 100644 index 000000000..c54c26fba --- /dev/null +++ b/openstacklib/lib/puppet/parser/functions/normalize_ip_for_uri.rb @@ -0,0 +1,31 @@ +require 'ipaddr' + +module Puppet::Parser::Functions + newfunction(:normalize_ip_for_uri, + :type => :rvalue, + :arity => 1, + :doc => <<-EOD + Add brackets if the argument is an IPv6 address. + Returns the argument untouched otherwise. + CAUTION: this code "fails" when the user is passing + an IPv6 address with the port in it without the + brackets: 2001::1:8080, to specify address 2001::1 + and port 8080. This code will change it to + [2001::1:8080] as it's a valid ip address. This + shouldn't be an issue in most cases. + EOD + ) do |args| + ip = args[0] + begin + if IPAddr.new(ip).ipv6? + unless ip.match(/\[.+\]/) + Puppet.debug("IP #{ip} is changed to [#{ip}]") + ip = "[#{ip}]" + end + end + rescue ArgumentError => e + # ignore it + end + return ip + end +end diff --git a/openstacklib/spec/functions/normalize_ip_for_uri_spec.rb b/openstacklib/spec/functions/normalize_ip_for_uri_spec.rb new file mode 100644 index 000000000..1b0dc5038 --- /dev/null +++ b/openstacklib/spec/functions/normalize_ip_for_uri_spec.rb @@ -0,0 +1,13 @@ +require 'spec_helper' + +describe 'normalize_ip_for_uri' do + it { should run.with_params(false).and_return(false)} + it { should run.with_params('not_an_ip').and_return('not_an_ip')} + it { should run.with_params('127.0.0.1').and_return('127.0.0.1')} + it { should run.with_params('::1').and_return('[::1]')} + it { should run.with_params('[2001::01]').and_return('[2001::01]')} + it do + is_expected.to run.with_params('one', 'two') + .and_raise_error(ArgumentError, /Wrong number of arguments/) + end +end diff --git a/tripleo/manifests/cluster/cassandra.pp b/tripleo/manifests/cluster/cassandra.pp index deca05fcb..b20926bed 100644 --- a/tripleo/manifests/cluster/cassandra.pp +++ b/tripleo/manifests/cluster/cassandra.pp @@ -52,7 +52,12 @@ $client_port_thrift = '9160' ) { - validate_array($cassandra_servers) + + # TODO: Remove this comment once we can guarantee that all the distros + # deploying TripleO use Puppet > 3.7 because of this bug: + # https://tickets.puppetlabs.com/browse/PUP-1299 + # + # validate_array($cassandra_servers) validate_ipv4_address($cassandra_ip) class {'::cassandra': diff --git a/tripleo/manifests/cluster/zookeeper.pp b/tripleo/manifests/cluster/zookeeper.pp index 6f4adbcd8..82d21eefe 100644 --- a/tripleo/manifests/cluster/zookeeper.pp +++ b/tripleo/manifests/cluster/zookeeper.pp @@ -39,9 +39,13 @@ $zookeeper_hostnames ) { - validate_array($zookeeper_server_ips) + # TODO: Remove comments below once we can guarantee that all the distros + # deploying TripleO use Puppet > 3.7 because of this bug: + # https://tickets.puppetlabs.com/browse/PUP-1299 + + # validate_array($zookeeper_server_ips) validate_ipv4_address($zookeeper_client_ip) - validate_array($zookeeper_hostnames) + # validate_array($zookeeper_hostnames) # TODO(devvesa) Zookeeper package should provide these paths, # remove this lines as soon as it will. diff --git a/tripleo/manifests/network/midonet/agent.pp b/tripleo/manifests/network/midonet/agent.pp index 2f1da9414..0e6528255 100644 --- a/tripleo/manifests/network/midonet/agent.pp +++ b/tripleo/manifests/network/midonet/agent.pp @@ -33,8 +33,13 @@ $cassandra_seeds ) { - validate_array($zookeeper_servers) - validate_array($cassandra_seeds) + # TODO: Remove comments below once we can guarantee that all the distros + # deploying TripleO use Puppet > 3.7 because of this bug: + # https://tickets.puppetlabs.com/browse/PUP-1299 + + # validate_array($zookeeper_servers) + # validate_array($cassandra_seeds) + # FIXME: This statement should be controlled by hiera on heat templates # project diff --git a/tripleo/manifests/network/midonet/api.pp b/tripleo/manifests/network/midonet/api.pp index 33b22178a..83efd2c74 100644 --- a/tripleo/manifests/network/midonet/api.pp +++ b/tripleo/manifests/network/midonet/api.pp @@ -78,7 +78,12 @@ $admin_tenant_name = 'admin' ) { - validate_array($zookeeper_servers) + + # TODO: Remove this comment once we can guarantee that all the distros + # deploying TripleO use Puppet > 3.7 because of this bug: + # https://tickets.puppetlabs.com/browse/PUP-1299 + + # validate_array($zookeeper_servers) validate_ip_address($vip) validate_ip_address($keystone_ip) validate_ip_address($bind_address) diff --git a/tripleo/manifests/ssl/cinder_config.pp b/tripleo/manifests/ssl/cinder_config.pp new file mode 100644 index 000000000..e1ed113a3 --- /dev/null +++ b/tripleo/manifests/ssl/cinder_config.pp @@ -0,0 +1,28 @@ +# Copyright 2016 Red Hat, Inc. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +# == Class: tripleo::ssl::cinder_config +# +# Enable SSL middleware for the cinder service's pipeline. +# + +class tripleo::ssl::cinder_config { + cinder_api_paste_ini { + 'filter:ssl_header_handler/paste.filter_factory': + value => 'oslo_middleware.http_proxy_to_wsgi:HTTPProxyToWSGI.factory'; + 'pipeline:apiversions/pipeline': + value => 'ssl_header_handler faultwrap osvolumeversionapp'; + } +} diff --git a/tripleo/spec/classes/tripleo_midonet_agent_spec.rb b/tripleo/spec/classes/tripleo_midonet_agent_spec.rb index eb9195e06..eb3abfe85 100644 --- a/tripleo/spec/classes/tripleo_midonet_agent_spec.rb +++ b/tripleo/spec/classes/tripleo_midonet_agent_spec.rb @@ -24,6 +24,7 @@ :hostname => 'host2.midonet', :osfamily => 'RedHat', :operatingsystem => 'CentOS', + :operatingsystemrelease => '7.1', :operatingsystemmajrelease => 7, } end From c15ac8caa3b0bda31d97a54b5267db6b9ec5f341 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martin=20M=C3=A1gr?= Date: Fri, 3 Oct 2014 19:18:45 +0200 Subject: [PATCH 29/35] Change default documentation URL Documentation should point to RDO docs by default. This patch is changing it. Fixes: rhbz#1092019 Signed-off-by: Gael Chamoulaud --- horizon/manifests/init.pp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/horizon/manifests/init.pp b/horizon/manifests/init.pp index fd404ea39..ce974a133 100644 --- a/horizon/manifests/init.pp +++ b/horizon/manifests/init.pp @@ -278,7 +278,7 @@ $api_result_limit = 1000, $log_handler = 'file', $log_level = 'INFO', - $help_url = 'http://docs.openstack.org', + $help_url = 'https://www.rdoproject.org/Docs', $local_settings_template = 'horizon/local_settings.py.erb', $configure_apache = true, $bind_address = undef, From 9e5a23f9578923a980edd917510ee9526bee65ae Mon Sep 17 00:00:00 2001 From: Lukas Bezdicka Date: Wed, 17 Jun 2015 14:38:23 +0200 Subject: [PATCH 30/35] [Rabbitmq] set repos_ensure to false We should use rabbitmq only from repos provided to us by RDO, RHOS or distro. Change-Id: I53ea9c10800846d5d0e6632b57e969159d64ff90 --- rabbitmq/manifests/params.pp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rabbitmq/manifests/params.pp b/rabbitmq/manifests/params.pp index 1101ebb36..1e73b31a6 100644 --- a/rabbitmq/manifests/params.pp +++ b/rabbitmq/manifests/params.pp @@ -69,7 +69,7 @@ $management_ssl = true $package_apt_pin = '' $package_gpg_key = 'http://www.rabbitmq.com/rabbitmq-signing-key-public.asc' - $repos_ensure = true + $repos_ensure = false $manage_repos = undef $service_ensure = 'running' $service_manage = true From f5099c271d835c91c357bc22dc5aa0c9e4c96d26 Mon Sep 17 00:00:00 2001 From: Lukas Bezdicka Date: Mon, 13 Jul 2015 17:39:39 +0200 Subject: [PATCH 31/35] Setup SELinux booleans if running in httpd SELinux booleans httpd_use_openstack and httpd_can_network_connect_db need to be turned on in order to have keystone working with httpd on SELinux enabled systems. Change-Id: I8722a4e619ac0d0a9e534e34721409d72e5c8323 --- keystone/manifests/wsgi/apache.pp | 18 ++++++++++++++++++ .../spec/classes/keystone_wsgi_apache_spec.rb | 13 ++++++++++++- 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/keystone/manifests/wsgi/apache.pp b/keystone/manifests/wsgi/apache.pp index 080e28c71..2e9bd1c0d 100644 --- a/keystone/manifests/wsgi/apache.pp +++ b/keystone/manifests/wsgi/apache.pp @@ -106,6 +106,9 @@ # (optional) Passes a string of custom configuration # directives to be placed at the end of the vhost configuration. # Defaults to undef. +# [*setup_selinux*] +# Setup SELinux ? (boolean) +# Optional. Defaults to true # # == Dependencies # @@ -155,6 +158,7 @@ $access_log_format = false, $vhost_custom_fragment = undef, + $setup_selinux = true, ) { include ::keystone::params @@ -164,6 +168,20 @@ include ::apache::mod::ssl } + if $setup_selinux and (str2bool($::selinux) == true) { + Selboolean <| |> ~> Service['httpd'] + # we need httpd to connect to keystone ports + selboolean { 'httpd_use_openstack': + value => 'on', + persistent => true, + } + # we need httpd to connect to databases + selboolean { 'httpd_can_network_connect_db': + value => 'on', + persistent => true, + } + } + Package['keystone'] -> Package['httpd'] Package['keystone'] ~> Service['httpd'] Keystone_config <| |> ~> Service['httpd'] diff --git a/keystone/spec/classes/keystone_wsgi_apache_spec.rb b/keystone/spec/classes/keystone_wsgi_apache_spec.rb index bad957613..4e5263fc8 100644 --- a/keystone/spec/classes/keystone_wsgi_apache_spec.rb +++ b/keystone/spec/classes/keystone_wsgi_apache_spec.rb @@ -271,7 +271,8 @@ let :facts do global_facts.merge({ :osfamily => 'RedHat', - :operatingsystemrelease => '6.0' + :operatingsystemrelease => '6.0', + :selinux => 'true' }) end @@ -285,6 +286,16 @@ end it_configures 'apache serving keystone with mod_wsgi' + + it { is_expected.to contain_selboolean('httpd_use_openstack').with( + 'value' => 'on', + 'persistent' => true, + )} + + it { is_expected.to contain_selboolean('httpd_can_network_connect_db').with( + 'value' => 'on', + 'persistent' => true, + )} end context 'on Debian platforms' do From 58adc04dab01b250e293bfd896e2ba6a85938177 Mon Sep 17 00:00:00 2001 From: Lukas Bezdicka Date: Tue, 8 Sep 2015 14:10:29 +0200 Subject: [PATCH 32/35] Remove trove ubuntu package hack Bug #1365561 has been marked as fixed so it should be safe to remove hack adding group and file before installing package. Change-Id: Id447a47a056a41d8c98b01d6769f4121a36d7aae --- trove/manifests/api.pp | 9 +-------- trove/manifests/conductor.pp | 9 +-------- trove/manifests/db/sync.pp | 1 - trove/manifests/guestagent.pp | 9 +-------- trove/manifests/init.pp | 31 ++++--------------------------- 5 files changed, 7 insertions(+), 52 deletions(-) diff --git a/trove/manifests/api.pp b/trove/manifests/api.pp index 401247186..4e2ba10dd 100644 --- a/trove/manifests/api.pp +++ b/trove/manifests/api.pp @@ -177,16 +177,9 @@ Trove_config<||> ~> Exec['post-trove_config'] Trove_config<||> ~> Service['trove-api'] + Package['trove-api'] -> Trove_config<||> Package['trove-api'] -> Trove_api_paste_ini<||> Trove_api_paste_ini<||> ~> Service['trove-api'] - # Trove db sync is broken in Ubuntu packaging - # This is a temporary fix until it's fixed in packaging. - # https://bugs.launchpad.net/ubuntu/+source/openstack-trove/+bug/1451134 - file { '/etc/trove/trove.conf': - require => File['/etc/trove'], - } - File['/etc/trove/trove.conf'] -> Trove_config<||> - Trove_config<||> -> Package[$::trove::params::api_package_name] # basic service config trove_config { diff --git a/trove/manifests/conductor.pp b/trove/manifests/conductor.pp index 0f9b4a52f..a8495600b 100644 --- a/trove/manifests/conductor.pp +++ b/trove/manifests/conductor.pp @@ -66,16 +66,9 @@ include ::trove::params + Package[$::trove::params::conductor_package_name] -> Trove_conductor_config<||> Trove_conductor_config<||> ~> Exec['post-trove_config'] Trove_conductor_config<||> ~> Service['trove-conductor'] - # Trove db sync is broken in Ubuntu packaging - # This is a temporary fix until it's fixed in packaging. - # https://bugs.launchpad.net/ubuntu/+source/openstack-trove/+bug/1451134 - file { '/etc/trove/trove-conductor.conf': - require => File['/etc/trove'], - } - File['/etc/trove/trove-conductor.conf'] -> Trove_conductor_config<||> - Trove_conductor_config<||> -> Package[$::trove::params::conductor_package_name] if $::trove::database_connection { if($::trove::database_connection =~ /mysql:\/\/\S+:\S+@\S+\/\S+/) { diff --git a/trove/manifests/db/sync.pp b/trove/manifests/db/sync.pp index 97d69dbfa..9d3f947e2 100644 --- a/trove/manifests/db/sync.pp +++ b/trove/manifests/db/sync.pp @@ -23,6 +23,5 @@ user => 'trove', refreshonly => true, subscribe => Trove_config['database/connection'], - require => Package['trove-api'], } } diff --git a/trove/manifests/guestagent.pp b/trove/manifests/guestagent.pp index 65d0740e2..aad16aef9 100644 --- a/trove/manifests/guestagent.pp +++ b/trove/manifests/guestagent.pp @@ -71,16 +71,9 @@ include ::trove::params + Package[$::trove::params::guestagent_package_name] -> Trove_guestagent_config<||> Trove_guestagent_config<||> ~> Exec['post-trove_config'] Trove_guestagent_config<||> ~> Service['trove-guestagent'] - # Trove db sync is broken in Ubuntu packaging - # This is a temporary fix until it's fixed in packaging. - # https://bugs.launchpad.net/ubuntu/+source/openstack-trove/+bug/1451134 - file { '/etc/trove/trove-guestagent.conf': - require => File['/etc/trove'], - } - File['/etc/trove/trove-guestagent.conf'] -> Trove_guestagent_config<||> - Trove_guestagent_config<||> -> Package[$::trove::params::guestagent_package_name] # basic service config trove_guestagent_config { diff --git a/trove/manifests/init.pp b/trove/manifests/init.pp index 57bc4013c..597392877 100644 --- a/trove/manifests/init.pp +++ b/trove/manifests/init.pp @@ -308,32 +308,9 @@ trove_config { 'DEFAULT/neutron_url': ensure => absent } } - if $::osfamily == 'RedHat' { - # TO-DO(mmagr): Conditional should be removed as soon as following bug - # is really fixed. On Ubuntu trove-common is not installable without already - # running database and correctly filled trove.conf: - # https://bugs.launchpad.net/ubuntu/+source/openstack-trove/+bug/1365561 - package { 'trove': - ensure => $package_ensure, - name => $::trove::params::common_package_name, - tag => ['openstack', 'trove-package'], - } - $group_require = Package['trove'] - } else { - $group_require = undef + package { 'trove': + ensure => $package_ensure, + name => $::trove::params::common_package_name, + tag => ['openstack', 'trove-package'], } - - group { 'trove': - ensure => 'present', - name => 'trove', - system => true, - require => $group_require - } - - file { '/etc/trove/': - ensure => directory, - group => 'trove', - require => Group['trove'] - } - } From a65d670ab32bba394c6b8bcb7f4ac5422525ced2 Mon Sep 17 00:00:00 2001 From: Lukas Bezdicka Date: Thu, 10 Sep 2015 13:11:55 +0200 Subject: [PATCH 33/35] Remove installation of pm-utils pm-utils was installed on compute nodes because of bug in packaging. This should be long fixed and pm-utils it's self is long dead in upstream. Change-Id: Ib5f2ee9b4ab58a32fd5f8c4ad514223795bd5225 --- nova/manifests/compute.pp | 4 ---- nova/spec/classes/nova_compute_spec.rb | 4 ---- 2 files changed, 8 deletions(-) diff --git a/nova/manifests/compute.pp b/nova/manifests/compute.pp index 8417d4a0d..81400b79d 100644 --- a/nova/manifests/compute.pp +++ b/nova/manifests/compute.pp @@ -219,10 +219,6 @@ } } - package { 'pm-utils': - ensure => present, - } - nova_config { 'DEFAULT/force_raw_images': value => $force_raw_images; } diff --git a/nova/spec/classes/nova_compute_spec.rb b/nova/spec/classes/nova_compute_spec.rb index ecea146bb..f0cd54b13 100644 --- a/nova/spec/classes/nova_compute_spec.rb +++ b/nova/spec/classes/nova_compute_spec.rb @@ -34,10 +34,6 @@ :before => 'Nova::Generic_service[compute]' ) } - it { is_expected.to contain_package('pm-utils').with( - :ensure => 'present' - ) } - it { is_expected.to contain_nova_config('DEFAULT/force_raw_images').with(:value => true) } it 'configures availability zones' do From 9dd0fed389ef306fa3dfffb50aa0619a50d28210 Mon Sep 17 00:00:00 2001 From: Alan Pevec Date: Tue, 12 Jan 2016 23:18:38 +0100 Subject: [PATCH 34/35] Dummy change to trigger OPM build BTW this docs seems to be out of date, gerrithub is used now at least for upstream-* branches: https://review.gerrithub.io/#/q/project:redhat-openstack/openstack-puppet-modules Also master and stable/* are pushed directly to github so their replicas on gerrithub are out of sync! Change-Id: I047001103356796a52d648f3165dd91457c026ef --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index b08066448..f9852e3e6 100644 --- a/README.md +++ b/README.md @@ -33,3 +33,4 @@ to the commit message, push the branch and [initiate a pull request](https://hel git commit --amend git push --set-upstream origin collectd + From 095fda822669faed5549ff4490082d6e70af962c Mon Sep 17 00:00:00 2001 From: Javier Pena Date: Wed, 2 Dec 2015 14:23:25 +0100 Subject: [PATCH 35/35] Follow-up on PyMySQL support for Red Hat platforms (Nova). Rely on packaging dependencies to avoid issues caused by different package names between Fedora and RHEL (python-PyMySQL vs python2-PyMySQL). https://review.openstack.org/#/c/245229/4/spec/classes/neutron_db_spec.rb includes all the discussion that led to this. Change-Id: I6ba84a2636411a0be76603d47ca09ee4c93fbfe6 (cherry picked from commit bd64e9274b33e89814a223240f3c954a71c8bca3) --- nova/manifests/params.pp | 2 +- nova/spec/classes/nova_db_spec.rb | 29 +++++++++++++++++++++-------- 2 files changed, 22 insertions(+), 9 deletions(-) diff --git a/nova/manifests/params.pp b/nova/manifests/params.pp index 29e9b0f95..df14e7e8b 100644 --- a/nova/manifests/params.pp +++ b/nova/manifests/params.pp @@ -27,7 +27,7 @@ $serialproxy_package_name = 'openstack-nova-serialproxy' $spicehtml5proxy_package_name = 'openstack-nova-console' $sqlite_package_name = undef - $pymysql_package_name = 'python2-PyMySQL' + $pymysql_package_name = undef # service names $api_service_name = 'openstack-nova-api' $cells_service_name = 'openstack-nova-cells' diff --git a/nova/spec/classes/nova_db_spec.rb b/nova/spec/classes/nova_db_spec.rb index 2a76aa0c5..1ef638931 100644 --- a/nova/spec/classes/nova_db_spec.rb +++ b/nova/spec/classes/nova_db_spec.rb @@ -79,12 +79,21 @@ } end - let :platform_params do - { :pymysql_package_name => 'python-pymysql' } - end - it_configures 'nova::db' + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://user:pass@db/db', } + end + it 'install the proper backend package' do + is_expected.to contain_package('nova-backend-package').with( + :ensure => 'present', + :name => 'python-pymysql', + :tag => 'openstack' + ) + end + end + context 'with sqlite backend' do let :params do { :database_connection => 'sqlite:///var/lib/nova/nova.sqlite', } @@ -108,11 +117,15 @@ } end - let :platform_params do - { :pymysql_package_name => 'python2-PyMySQL' } - end - it_configures 'nova::db' + + context 'using pymysql driver' do + let :params do + { :database_connection => 'mysql+pymysql://user:pass@db/db', } + end + + it { is_expected.not_to contain_package('nova-backend-package') } + end end end