Jump to content

Portal:Cloud VPS/Admin/notes/Puppet Possibilities

From Wikitech
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Talking Puppet, Hiera, Openstack and process

Meeting Agenda

Old universe

  • Nova in many places for (nearly) all things
  • intermingled repo management
  • Current install specific groupings (nova-api and nova-network, etc)
  • Per host values for labtest in hiera and regex madness
  • hiera lookups buried in: roles, modules, templates
  • Overlapping requirements for components handled via require_package() a lot
  • Passing dicts of huge numbers of unrelated params to get to a few
  • Duplicate params at various levels of the hiera tree
  • Convoluted component interdependence
  • Associated configuration lumped into large blocks in roles or other
  • ferm rules that make it difficult to reason on cleanup as it's unclear why it existed
  • Inconsistent strategy on hostname => ip esp for ferm
  • data structured mingled in with code (labsdb, etc)
  • The same value would be named multiple things throughout a config tree

General

  • Three failover scenarios
  • labcontrol (host and cold) via hiera
  • controlled via hiera params setting a boolean in a profile and passed to modules 
    class openstack2::keystone::service(
    $active,
  • labservices (partial host and cold)
  • labnet (host and cold) role swapping <== looking to deprecate


  • Deployment separations (deployments to come)
  • deployments may end up being regions?
  • deployments are made up of components
  • components can be upstream openstack specific or not
  • Why not use inheritance:
  • We are working on a holistics openstack / VPS product profile model
  • consistent naming across components glance/service.pp, keystone/service.pp
  • If it is parameterized per deployment is worth doing base and deployment profiles potentially under profiles/manifests/openstack
  • Understanding our Hiera use cases:
  • Labs hiera tree is (also) a nightmare.
  • expand_paths vs not
  • labs/private is an overloaded dual purpose shim
  • back end split
  • Labs hiera is a nightmare
  • Per deployment labs/private or equiv may be needed
  • Puppet norms for new style are not always set up for sucess
  • base_firewall, standard, remaining implicit override, hiera adding functionality, etc

Upstream to us Style


To Be Done

  • Issue of cross site deployment variables (db1047)
  • Issue of special failover with labnet
  • firewall rules buried in site.pp
  • tools stub zone on labservices persistence
  • Issue of cleanup for hiera
  • openstack2 => openstack
  • test failover for pairs

Moving forward

  • resolve() within ferm
  • per ferm update resolution instead of per puppet run
  • human readable in diffs
  • Our own sub-style consistency that is an extension but not an override
  • File and resource consistence etc
    • owner,group,mode,source,requires
  • manifest flow: include, package, file(s), service
  • This results in per deployment per component profiles and per deployment roles
  • Much easier to test in labtest without effecting main, and we SHOULD/need to/must
  • staging models mean more of an opportunity for productive meaningful refactor
  • testplan designations
  • "foo: bar for x". "testplan: baz?"
  • modules/roles/manifests/wmcs
  • potentially: modules/roles/manifests/wmcs/tenants
  • Anything
  • Functional approach to hiera and parameters. Ironic? :)
  • In the mid-term we must consolidate on WMCS managed hiera strategy
  • A large and intimidating Toolforge cleanup must happen
  • We will have to solve the puppet multi-tenancy problem in the next 2 years or die trying
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Portal:Cloud_VPS/Admin/notes/Puppet_Possibilities&oldid=1851241"